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The purpose of this work was the development of a technical data file 
concerned with the technology for Instructional System Development (ISD) 
suitable for a variety of users. The file was to be prepared In a way 
amenable to later computerized storage and retrieval* 



Approach 

Considerable initial attention was given to planning and testing the 
major processes Involved. These are (1) identification of items for 
Inclusion^ (2) abstracting individual documents » (3) decisions regarding 
indexing^ formatting, and other particulars of file development, and (4) 
production of the file. In order to accomplish the selective search 
(necessitated by the limited nature of the effort) the best procedure 
seemed to be to review general information sources and indexes of highly 
probable relevant cohtent with key words and also to search relevant 
specialty journals covering the period 1950 to June 1973, Abstracting was 
accomplished by a technical team which had (as individuals) received care- 
ful pretralnlng in the designed procedures and which was continuously 
monitored as to output quality* A detailed handbook or guidelines on spe- 
cific procedures was available for training and for use during the work. 
Abstracts of articles providing opinion, new methodology, evaluative sum- 
mary, and literature review were of a summarizing descrlptlve-fiature* 
Abstracts of articles reporting sampling studies, correlational analysis, 
and experimental data were prepared more comprehensively so that, for many 
purposes, *the abstract could be used in lieu of the article, A common for- 
mat was used with a bibliographic/indexing information page and an evalu- 
ation checklist being Included in addition to the main abstract. Thus, 
provisions were made for convenient storage and retrieval and comprehensive 
coordinate indexing. Production was via IBM MT/ST so that hard copy as 
well as tapes readily convertible for computer input were prepared. Special 
procedures for preparation were followed to ease the difficulties of any 
later computerization, 



Results 



The principal results of the effort are as follows: (1) a paper file 
of 2,693 abstracts, (2) a card file of titles, (3) a coordinate index, 
(4) a comprehensive key word index and bibliography, and (5) MT/ST tapes of 
1,950 of the abstracts, Incidental to the development, comprehensive 
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guidelines for abstracting this type of literature and a compendium of 
author-noted research-and -development needs were prepared. 



Conclusions 

Although not a complete file on instructional technology, the file 
developed does represent a considerable proportion (perhaps twenty percent) 
of the relevant literature* Furthermore, by selectively including the 
recent literature of 1950 onward, the utility of the file is enhanced. The 
comprehensive indexing and evaluation coding still further improves the 
immediate usefulness as well as the potential for using the file as the 
basis of a computerized system. In short, the file represents a good start 
toward a complete information system on instructional technology. 



This summary was prepared by T. E. Cotterman, Training 
Technology Branch, Advanced Systems Division, Air Force 
iHuman Resources Laboratory 
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SECTION I - INTRODUCTION 



Purpose 



The purpose of this effort was to prepare a comprehensive set of 
descriptive and Informative abstracts of the literature related to the 
technology of Instructional Systems Development (iSD)i It was planned 
that the abstracts might later be used as input to a computerized technl-* 
cal data file that will provide prescriptive information for ISD personnel 
(Including researchers, manager and designers of Instructional systems) « 
Until that time, the abstracts will be used in a paper-file format. 



Background and Discussion 



The technology of ISD evolved over the past two decad^es through the 
efforts of learning theorists, educators, educational researchers, and 
human factors researcher^. In each phase of development the technology 
has been applied with great vip,or (and varying amounts of success) to 
military, industrial, and educational learning situations* 

Perhaps in part because of the variety of its foundations and appli- 
cations, the gulf between Instructional technology and its implementation 
continues to persist to an unconscionable degree. Although many problems 
in instructional technology remain to be solved, much of the necessary 
information is available in the relevant technical literature. Ready access 
to such information is needed not only by the designers and managers of 
instructional systems, but also by the researchers whose efforts may con- 
tribute to the refinement of the technology. 



Sources of Information 

Reference to the information is found in a variety of sources and 
locations. In many cases, practitioners have little time available for a 
comprehensive search for available abstracts, much less for the acquisition 
and study of the original documents — an activity that is often required 
because the available abstracts do not contain information to expedite the 
decisions of either the instructional system designer or researcher. 

The literature relating to the technology of ISD is varied* The major 
types of documents are: 

1. Opinion articles 

2. Methodological developments 
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Bvaluative summaries 



4. Literature reviews 

5# Bibliographies 

(6, Statistical sampling studies 

?• Correlational research studies 

8. Research studies in which variables are manipulated 

User J^eeds i 

Depending on their particular needs and the amount of time available, 
designers and managers (practitioners) as well as reseaichers may need 
information from any or all of the eight kinds of documents. To a large 
extent, properly prepared abstracts of research studies can eliminate the 
requirement for returning to the original documents. Abstracts of the 
other types of documents can greatly facilitate the decision as to whether 
the original document should be viewed. 

Although there is considerable overlap, the needs of the instructional 
system designer and the researcher are not always identical. The de.signer 
and managers must develop an operational system in the face of a host of 
resource limitations and time constraints. As practitioners, they encoun- 
ter, for instance, predictable problems for which specific guidance is 
required (e.g., the development of objectives, sequencing of objectives ^ 
selection or development of media). Descriptive abstracts stating the 
nature and scope of a specific document will serve the practitioner's 
needs — if the abstracts and original document are appropriately indexed 
(accessible), and if the document is evaluated in terms of the practi- 
tioner's needs. 

Researchers, especially during the problem definition and hypothesis 
formulation phases of their work, need more detailed abstracts that de- 
scribe the purpose, method, results, and supportable conclusions of related 
research work. The researcher needs informative abstracts of research 
studies which concisely present information of high probable significance 
in the original record. When combined with an appropriate evaluation, such 
abstracts obviate the need for the researcher to obtain many original 
source documents. And, they are also useful for the practitioner who 
requires this level of detail for his instructional system design activities. 

Finally, all types of users can use prescriptive information of the 
sort contemplated for the computerized file* With appropriate coding and 
indexing the computer can search for relevant data items from both descrip- 
tive abstracts (prepared primarily for the practitioner) and informative 
abstracts (prepared primarily for the researcher). For example, an English 
language (i.e., non-computer) version of a user query to the computerized 



file tnight be: ''List sutttoarles of all methodological development articles 
(evaluated as useful for ISD) that dlacuds the use of color as a medium* 
Also I list conclusions of experimental studies (only those judged methodol** 
ogically sound) that investigate the use of color as a medium*^^ 

This example » representing selective retrieval » only hints At the 
potential uses of a prescriptive search-and-retrieval system. For instance^ 
a user of the prescriptive file might query ao follows: ''Tabulate the 
results of all methodologically sound experimental studies published since 
1965 which found that the use of color had a significant effect on learning*" 
Or, going one step further, a user Could query: ''Given the following char- 
acteristics of the learning task, does the literature Support the use of 
color as a medium? Obviously, such queries require extensive coding and 
indexing of the abstracts, as well as a sophisticated search-*and-retrieval 
system* 



The Status of the AFHRL Technical Data File 

Although the implementation of a prescriptive search-and^retrieval 
system remains to be accompliahed, the necessary ingredients exist < The 
feasibility of the computerized file has been demonstrated ^Reference 
Number 6)» The present effort has produced comprehensive abstracts of 
approximately 20 percent of the relevant technical literature and all these 
appropriately indexed and coded for use in a computerized file* Until a 
computerized search-and-retrieval system is implemented, the file will be 
available in notebooks in a form that can be easily converted to computer 
inpu?;. 



Report Overview 



The remainder of this report describes the methodology used in liter- 
ature search, the process used in preparing the abstract, and a summary ^of 
the results of the effort. The report concludes with three appendixes* 
Theiifirst contains examples of a descriptive and an informative abstract* 
Appendix II contains the special typing requirements for preparing abstracts 
suitable for computer input. The third appendix lists the references for 
A5 specific journal articles which are relevant to ISD but which were not 
abstracted* 
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SECTION II - METHODOLOGY 



^ In t r od uc t ion 

.'■ p*^ ■ ■ ■ ■ ' ' ' " 

This section describes the four major processes followed in developing 
the technical data file: 

1. The literature search 

2. The abstracting process 
3* The file development 

A. The production of the file \ 

The development of these procedures was facilitated by earlier work in 
abstracting material for the Handbook for the Designers of Instructional 
Systems (Reference Number 7). 



The Literature Search 



Overview 

It was estimated that abstracts of 10,000 to 15,000 documents could 
reasonably be prepared as part of a technical data file on the design and 
use of instructional systems. However, it was clear from the outset that 
the current effort could only scratch the surface of the relevant liter- 
ature. In fact, about 20 percent of the potentially relevant documents 
were abstracted. The objective was, therefore, to search for and select 
documents of highest potential benefit to users of the file. 

The following general guidelines for the search were established: 

1. Establish a list of search terms (key words) relative to ISD. 

2. Search general information sources of high probable relevant 
content first. Such sources offer a number of advantages: 

a. They are usually comprehensively indexed. The 
index provides a quick method for examining a 
large number of document titles and search teras. 

b. They sometimes contain mini-abstracts that facil- 
itate selection of relevant documents. 

Cumulatively, they provide a reasonably comprehen- 
sive listing of titles produced since 1950. 

11 



c. 
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3. Search specific journals containing a reasonably high pro- 
portion of relevant articles and studies. Search the most 
recent volumes first, and work backwards in proportion to 
the payoff In ^'^ch journal^ 

At the conclusion of .ue effort, a general-level search covering 1950 
through June 1973 was completed* Furthermore, a detailed search of the 
most recent volumes of relevant Journals was completed, and some journals 
providing especially useful studies or articles were abstracted from 1950 
to the present* 



The Search Terms 

Early in the project, a list of search terms relevant to ISD was 
developed* The development of the listing was conducted in four stages: 

1, The *'first cut" was developed from existing key word lists 
and from specific, highly relevant documents. The sources 
used were: 



a. Subject indexes from Tufts Human Factors Bibliography 
(Reference Number 5), American Psychological Associ- 
ation, and Educational Resources Information Center, 

b. Air Force Manuals 50-1 and 50-2 (Reference Nurabers 
2 and 3). 



c. Department of the Army CON Reg-350-100-1 (Reference 
Number 4) . 

d. Air Training Command Library, XPTIA, Training Re- 
search and Development Directorate. 



2. The list was expanded to include media-related terms gleaned 
from the review of specific documents. 

3. The list wrs further expanded to include common experimental 
variables gi-^aned from the review of specific research 
studies . 



4. The final list was reviewed for consistency with sources. 
Redundancies were eliminated, and potentially confusing or 
ambiguous terms were defined. 

The completed list of over 600 common search terms and their definitions are 
given in the abstractor's guidelines (Reference Number 9). 
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Sources Searched 



The list of search terms was used to key the search of the 11 general 
information sources in Table 1. The general sources were employed for 
three purposes: 

1. To find relevant general documents and texts for which 
descriptive abstracts would be prepared. 

2* To find government and commercial agency technical report^ 
of research studies for which comprehensive informative 
abstracts would be prepared. 

3. To build (from relevant titles) a preliminary list of 
journals that probably contain relevant articles and 
research studies » 

The list of 48 journals searched during the project is shitm in Table 2. 

Searching entire journals offered two advantages over expanding the 
search of the general sources: 

1» The comprehensiveness of the search was insured. Search of 
specific journals facilitated bookkeepings reduced redundancy 
in the search effort and permitted clear definition of the 
extent of the search, ^ 

2. Specific competencies of each individual abstractor could 
be compared at this early stage to the subject matter and 
to the levels of statistical and technological sophistica- 
tion of each Journal to be abstracted* This permitted 
abstracting assignments that insured the abstractor's full 
comprehension of the material to be covered* 

The journals iu Table 2 were generally searched In the following 
manner, Titles and abstracts (when available) of 1970-1972 issues were 
reviewed* Next> the preceding three years' issues of each Journal were 
reviewed. Finally, the process of reviewing back issues in increments of 
three years was continued only for journals generally exhibiting a high 
proportion of relevant documents. 



The Selection Process * . 

The major inputs for selecting relevant documents were the titles of 
reports and journal articles, and their associated descriptive abstracts 
(when available). The following five-step selection process was used to 
select articles to be abstracted from the list of titles and abstracts, 

STEP 1: Reject the document if it is primarily concerned with: 

a. Animal training 

er|c 



Table 1 
General Information Sources 



SOURCE 


YEARS 


Human Factors Bibliographic Series (Ref, Number 5) 


1950-1965 


Other Bibliographies (e.g.. Reference Number 11^ 


1950-1965 


Organizational Bibliographies (AIR, ASA, HumRRO, 
LKDC) 


1950-1972 


Miscellaneous Sources (e,|g., Reference Number 12) 


1950-1971 


ERIC/CUE 


1966--1972 


Defense Documentation Center 


1950-1971 


Psychological Abstracts 


1966-1971 


Annual Review of Psychology 


1950-1972 


Bibliographic Index 


1950-1972 


Education Index 


1950-1972 


United States Office of Education (USOE) 
Research Reports 


1956-19?2 
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Table 2 
Journals Searched 



American Behavioral Scientist 


oournaX ol dxperxmcnkax cAiu^avxuii 


American Education 


journal oi uenerai ro/cnoxogy 


American Educational Research Journal 


Journal of Programmed Instruction 


American Journal of Psychology 


Journal of Psychology 


American Psychologist 


Journal of Reading Behavior 


Audiovisual Instruction 


Journal of School Psychology 


AV Communication Review 


Journal of Verbal Learning and 


California Journal of Educational 




Research 




Comparative Education Review 


Perceptual and Motor Skills 


Contemporary Education 


Personnel Journal 


Educational and Psychological 


Personnel Psychology 


Measurement 


Psychological Monographs 


Educational Record 


Psychological Record 


Educational Research Bulletin 


Psychological Reports 


Educational Screen ana Auaiovisuai 
Guide 


Psychological Review 


Psychology in the Schools 


Educational Technology 


Quarterly Journal of Experimental 


Harvard Educational Review 


"P o \f /> M 1 r\(^\3 

1; oydiuxuj^y 


Human Factors 


Kesearcn xuarterxy 


vjuumax ui Appxxeu^ DeiioVAur micixjroxo 


Review of Educational Research 


Journal of Applied Behavioral Science 


School Management 


Journal of Applied Psychology 


School and Society 


Journal of Communication 


Training and Development Journal 


Journal of Educational Psychology 


Training in Business and Industry 


Journal of Educational Research 


USAF Instructors Journal 


Journal of Experimental Analysis of 




Behavior 
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b* Training children 



c. Unusual learning environments such as: 

(1) Weightlessness 

(2) Under water 

(3) Centrifuge 

d* Unusual trainee states such as: 

0) Fatigue and sleeplessness 

(2) Drug states 

(3) Brain damage 

B* "irrelevant" tasks such as: 
(1) Music ~* 



STEP 2; Reject the article unless it is related primarily to instruction as 



reflected in the ISD process. This Includes determination and 
specification of training requirements, operationalization of 
training objectives, development of testing procedures, develop- 
ment of instructional methods, validation of instructional methods, 
application of instructional technology, and evaluation of the 
instructing system. 



STEP 3: Reject the article unless the title or abstract contains or implies 



the following positive identifier terms or cognates* (NOTE: These 
ppsitive identifier terms were selected to cover the five stages of 
the ISD Model in AFM 50-2* See Reference Number 2*) 

a. Systems approach to learning .(Instructional Systems, 
Instructional Systems Development, Systems Approach to 
Training) 

b. Task analysis (Job Analysis) 

c. Simulation training, 

d. On-the-job training 

e* Training equipment (Training Devices) 
f. Performance testing 



(2) 



Art 



(3) 



Maze learning 
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Skill training (Skill Acquisition) 



h. Computer-managed Instruction (NOTE! CAI Is con- 

sidered here as a subset of CMI) 

i. Media (Audiovisual Media » Multimedia, etc.) 

j . Educatlonil technology (Training Technology) 

k. Programmed instruction 

1. Strategies for instruction 

m. Teaching machines 

n. Group pacing 

o. Student-centered instruction (LCI, Self-Instruction, 
Self-Paced Instruction) 

p. Individually prescribed instruction 

q» Field evaluation 

r. Job performance evaluation 

s. Instructional equipment 

STEP As Divide remaining titles having potential for abstracting into two 
groups: 

a. Those that are definitely relevant* 

b. Those which may be relevant. 

STEP 5: List titles (and short abstracts when available) for all doubtful 
items and conduct a staff discussion to decide whether or not to 
abstract* Consult AFHRL as necessary. 

As the staff gained experience, it was discovered that this process 
did not filter out enough of the marginally useful documents. Articles 
with catchy titles which tended to rehash well-established principles were 
falling through the net and being assigned to abstractors. But, it was 
found that if the selection criteria were made stringent enough to exclude 
these articles, many truly valuable ones were also eliminated. Because a 
few of the specific journals proved to contain most of the relatively use- 
less articles, they were eliminated as primary sources* In this way most 
of the Journals that were searched yielded a reasonable proportion of 
articles and studies relevant to instructional system development* 
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The Abstracting Process 



In order to prepare 2,700 technical abstracts in accctdance with the 
project schedule, a proceduralized production process Vvas required, supple- 
mented with extensive quality assurance activities. To provide a clear 
understanding of the abstracting process used, the following topics will 
be discussed: 



1. 


The two types of abstracts 




2- 


The abstract form 




3* 


The selection and training of 


abstrac tors 


4. 


The assignment of articles to 


abstractors 


5. 


The preparation of acceptable 


abstracts 



Types of Abstracts 

Two general types of abstracts were prepared during this effort: Type 
I (descriptive) abstracts of special value to designers, and Type II (infor- 
mative) abstracts primarily for use by researchers. 

Type I Abstracts > Type I abstracts were prepared for opinion articles, 
methodological development articles, evaluative summaries, literature 
reviews, and bibliographies. In contrast to Type II abstracts, Type I 
abstracts cannot take the place of the document. They summarize the con- 
tents sufficiently so that a uset of the technical data file can determine 
whether the original document is appropriate for his needs. These abstracts 
Include a statement of the interrelationships among topics, and their rela- 
tive emphases. Within the confines of reasonable space limitations, items 
in the document which are of particular relevance for the instructional 
system designer (e*g., a short procedure for evaluating the effectiveness 
of CAI) are noted. Type I abstracts also include an evaluation of each 
document's clarity, readability, and unique contribution to the ISD tech- 
nology ♦ 

Type II Abstracts . Type II abstracts were prepared for statistical 
sampling studies, correlational research, and research studies that mani- 
pulate variables* Type II abstracts are sufficiently detailed that a user 
of the technical data file need not go back to the original document 
(unless he has an unusually detailed requirement, such as actually repli- 
cating the research that was reported). Type II abstracts include a sum- 
mary of (1) the problem posed by the investigation, (2) the method, (3) the 
results, and (4) the discussion/conclusions sections of the document* The 
last major section of the Type II abstract is an evaluation, which Includes: 
(1) a consideration of the validity, overall clarity, and appropriateness 
of the research design; (2) an evaluation of the appropriateness of the 
conclusions; and (3) a statement about the relevance of the work for ISD. 
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The Abstract Form 



Examples of a completed Type I and Type 11 abstract are In Appendix I. 
Both types of abstract were prepared on the same general form« The form 
was designed with two purposes In mind: (1) It permits the extensive 
coding required for computerizing the flle> and (2) It allows temporary 
use of the abstract In a paper file* Portions of the form are appropriate 
to every abstract. Other parts are relevant only to particular documents 
having specific characteristics A blank Abstract Form (for the hand- 
written abstract) Is presented in Figure !• In the following paragraphs, 
the numbered subsections of Figure 1 are discussed « 

P age 1 of the Abstract Form ("First" Page ) 

1. Sequence Number > The sequence number was assigned to identify 
the order of document acquisition! It is also the number used 
in the bibliographic master list» (See Reference Number 8.) 

2, Abstract Type > The abstractor checked off whether a Type I or 
Type II abstract was prepared. 

3' Reference Information . This section^ filled in by the typist > 
is based on the bibliographic reference cited by the abstrac* 
tor on Page 2 of the Abstract Form. The information was 
repeated on the firyt page to facilitate coding of the individ- 
ual bibliographic items (author, title, publisher, report 
number, and date) for computerization of the file. 

A» General Search Terms . The general search terms were assigned 
by the abstractor after reviewing the-article. These words 
constitute the primary means of search for the abstracts. 
The possible search terms have been compiled into a Directory 
of General Search Terms. This directory (and definitions of 
the terms), presented in the abstractor's guidelines (Refer- 
ence Number 9), helped abstractors use standardized tenol- 
nology* 

5. Unique Search Terms . Unique search terms, assigned by the 
abstractors, are words that are specific to a given document. 
For example, they may be descriptions of major systems 
involved, trade names for particular types of media used or 
discussed, new terms discussed at length in the document, or 
the names of authors discussed in detail in the document. 

6. Suggestions for Additional Research . The authors of docu- 
ments often mention suggestions or implications for future 
research and development, These suggestions or implications 
weri^ summarized by the abstractor. A compendium of these is 



'^OTE: The typed abstracts in the Appendix were prepared from special 
typing instructions (See Appendix II) and differ slightly In format 
O ^T^om the original abstract done by the abstractors. 
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AUTHORIS): 

mil. 

PUBLISHER: 



\ StQUtNCE NO, 



AeSTflACT TV^£; » II, 



ABSTRACT FORM 



REPORT NUMBER; 



GENERAL SEARCH TERMS: 



DATE: 



(?) 



UNIOUESEAnCK TERMS;. 



- ® 



SUGGESTIONS FOR ADDITIONAL RESEARCH: 



® 



CnOSS REFEn£NC€: (SEQUENCE NUMBERSI 
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Figure 1. Abstract Form 
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SEQUENCC NO . (l) 

BIBLIOGRAPHIC ReF€RENCE: . (§) 



BODY OF ABSTRACT: . (g) 



EVALUATION OF DOCUMENT: = (g) 



[NUMBEA or PAGES ! « 

(NUMBEROF REFEAENCES *f@ 

{INITrALS OF ABSTRACTOR >| 
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$uMfMAy anS evaluation ronM 

f I. CHECKUST$FORIVALUATINQ 
' TYPE I & TYPE II DOCUMENTS 
JA. IN$T«UCTfW$ 



• ittn .Tt>*i MM A«t«0P'* Ln t«r *>•». '*^*'t 1^1 Ml lA IM tottr* 
I >t t «'>r«i>'i<"r ttiy til, Mii|t M in >ii<« t^tir^o'*. i^i 




fVALUATiON CHECKLIST FOft TVPf I A8STftACT$ 



OltCf>ptrvi Surr^mjrv oi Contint 

ft. ^r«til|m«<l* liiO«irM»vM4u|t«iV. 

4 iipdr&K'k II »^»«>>m I'M II M>P«»pr<l<» 

I. 4^ *Wu>rifitl««tM( ri>t«»At<lrtfttwr*ltf(»tM. 

Id. nB«>twi ll >«i««nN I'll^ f^^loMlW 

n. >i«**'i*i »p«^*v 4««vt i^pitvid. 

11. C«n€Ki»teA« irl lUtM c^M^'v 

II tv%^^>4»^tH irl |kib|f*AM|CM»v •«itfw%<i prM*nt«d Irt IM «0<u«*k»«^t. 
If C&/>tlwa>|nt Kt IfrnllAM T« »t f^tm •>«• tfttcuMd bo thi llWult^V 
li. ll' ^MtNi'l 'IMyih irt i««kiiiv fa'fH^VM. 

Ci»«ii> U>»A>lf^»»i 

It. 004um«>M pi|»l«M» lp<>IK;ttl«nt l|^»vtn1 Id itO 

9^. £«A<lvtlaA4 C'l iw»4<itn| itO 
3} I' wpf'tlpf <• tPp'SP^Itl fft* tvP'Cll '>D ulW» 

34 Oa<wrt<^4 <« rMMni^'r iMv w tcHaw lAd ^ na« co'«>uiiA| 
31. fona •■ tt^a d0<u/^•^l d^i»<ai» an Mnb>M*». k'^«irtia« K a^n^** «tiit\<d« 

IC tVALUArtONCMfCKUST fOR TV^C II ABSTRACTS 

1. TitH 9* ^uT^tnt atfMv.|lt<v datC'iOai conTMOtV 

3. P>ot>«m it nittij c<«ariv. 
3>. ffaaaont *«' 1^ ait 't«t lAg Oft >*m tr* cia*». 

4, Ait«>fT^<la««t«'at<at«<ic>«ar'«. 
ft, i.i<fiita(ia<>« af H^a itudv ata Mit*4 
ft i<tH>»Mant tat«"a ara d«ti(^*d. 

f. Mwatianahip tha »'«bia<n to p^avio^i ra*a*'(^ i« (laaf. 

^r»(i4var««a«'cA a raK<*<k«d ida^vitti'i *s< auTt^'i aii)«<>ta». 
I, HTi>otMaa»aiattatadc>«*riy. 

lotaii^t. iga. a^d «tf«ar 




crnij"- 



^::ITJ1^ 
TO 



3^ 



nxn 

m 

.•-.-rn 
r.T:LTJ 



4i lAf«V)tV«r>4 r«l1«l »«A«*wil»r>i fl i^Mrlv tpkf >M li fhcii 
H. Inlar|ht<*l 1^4 >' .trr rttlo^ IftrvKdli |ni »r«|Wilit<M 
4>, ftMrwlallo^l •4 t^«oH|M| |ri isot ft^iia^tid ii can(iu«i*>ta 

thnt«fi«#*fa'»» ;**4at* 
[Mil 4C ifT<d4icilk»M f«r :uturi rM*arcM'l >*ft^ii<* ta'mu'atad. 

CUritv ird U^^fuln hi: 

•leo'i IV *f'tiii«i«aii» 
^ AaOOM i«iift»<iMT nianifa^. 

A3 Ta"! tH«'«o«'t leiavt i^ u"»^im4. iiT«04riiai KitAt>fic iwtuda. 
^ 'I a( im»^la«Hi \« pt«t**^¥94. 



2. DOCUMENT X TOPIC 
PROFILE 



14A 




^of^/atioA r« k<laqk>aia<* 4«ac*««d a* M i^ii 
tttfac'trHtitt 'ai«v*At r* raaaa'c^. 
Mait««d at w^tirof If idaqua^Wf d«a<r>»«4 aA4 x i^a^f uta. 
%if,fi\» >« idaquaiai* dtacritiad »« le i>rii »«eaii»N. i«aL a^ •I'^i* 
r»t«vt/tl c*t|r«ctafitt(Ci 

c«i da»'g^ ia s«acr>t>«d Ki^iy 
Kit'«'Mii '»» io«c.t>c '«»**/c'« itadnuitarv )uit»*M. 

MataaxK 9*«ifA mi M'lail '«M ioK.t.ft<% of tha |t«Dt>iam. 
ri>« al^Kfi ot iAd«p4n<i«/<t «iri|6r«a vi A«< centoofvj*4 Mith th« 
•4tact«a< urtcoNirg^lad. aai'i^*»u« ahlaa. 
toa a^acii a' l'>o•«>a<^<l«M •■riabia* a^a j«at co<if»<jnd*d #i<iift tM 
ai'«<n i> A^oiKaT'pnai »a/iabi««. 

C<K^t'a> fl<auip4 tfa ap^^qpo^ta to« ca^Vianton «tith •> m« imaAta* 

in<»p*f><lant«a'libf*» I'l KJwqua^i^V U>4c(fi«<l lUua'^'liiMfv) 
^*c>vM»%nt «t«>»b'-M iiaa4w%.att«» n>«t»<ia4 tOv,a»iwt»iM 
Vtf^aa'at a^a »o«>»t.«naiN a«iiA»d. rO^janT'liiivi. ««n«r»iivl 
ApvxHJfiaia iit>»«iu>M i>i M<aci*d lo« dapan^afM vaftab'M 
t •o««<'<^ia' 'naropv'am oM< M Mvi ><• rnankpuratH. 
Ctv'dtttaint tjAci* aa|-*r'f^»"t MM cafo^vKtad i<ac<aarrv ipacUtad 

•nd<Ja><'<«af#<^ •'I *ut*K<t>\» (Jata.f 
R«M»>^n'da«'«A 1 'rai o» ai^at wae'** 

c»'» »«(haf .fa '»*«'«>oi a* »'0<a<5v<i» i'» 4a*f bad »u«t<iani *aia*t 
tA ido** *a(i<xat<«n. 

Oaifraa^t^''** -r^ifr** »« »«a<*du*a» •tt'io'* <f'i < a*!* « h 4«a<t«, 
rt^t)««d« IX ^B<.a4ii^i4 I'l aptwoc'Ki *i 1^1 iCuikart al i^a l»t«biam, 
Of 'w^ft* »* P'of,K'm iJp iA«i»««ti arntaetvH biaaaa 
lAia data. 

n«*a«ihar h»* (ak »« ipcsc'iata itapa » aiimk>ari «a«t'0< i»r 
a<(>Miiir«oiaf b'M and furfil»i««t 

30 Oa<aSii>s«rkr>tfna(t\o4i v p«aca4w'Bi«'i «^iM cooKxiv 
H. »! maa«u>1n« Inal ru rtt«^ n H Mtab*ilM4 

3*. va">4 »» a' rt^^aavfir^ >MlfulT^♦«lJ la N'abn^haA 
iX Apo<oo'-a"a <^«iNk»i wari«viaTM da(a a«to»*)a"«a 

Miv^ t^iM indjcaiad bv rt<a 'ftaaat^ tfaaJ^s. 
14. MtiKci(H^«*d la arva^vaa tP^a daia ad <i«a4 ift/>«ciir 
3V KmoIi4»' iM a<^alv••• fl »««Mn(a4craa(i«. 
M. dfaciM a«<d labta* a^ |hi«a^fa<f (fa«rTv 
l>. On)» i<r<'*<l^* 'laulu iia i««tta^ad M in«M>Uvfft^i 4ata. 
34 ConrKianca l#«tt« «<a itaiad <Wi« ti^ u«s<lbi*). (uttiriad. 
3ft. U'«wit^a< Katklical p«b<.»<^r*t »< unuauaf a<>«1lcatt*M af ttti^^i 

broiadufM aia daicKtatf ctaaTy »fvd |u«iti*d. 
•«> rr»n,ferm*(l dali I'l (iaa<iv ^>ac>IM4 ai ku«M a(v4 «ra«h«d« »f caovati^oA 
•/*4{af*fl *ia*^!» a<»tf a<>a<Kata»^ ^t*i*'»^- 

a) C«ncK«iJo^« *»a itatad c>aaffv. 

d). C«>^cK<tJoJH ara iub«tam^aiad bv tha avkMf^a »»a*awwd. 
CJ, G*r^¥»nfXief>» *ra c»«»lr%a<l ta «ha ♦aoMla»*«»o »m kfctKa tampJ* 
#iaa tfraiMrA, 

44. C*v>«'<*a«»« 'a<ata ta tha »ara«wtar* Uh/«tti$tis»«* i*>#» 
M*B «>aK;^a4 wtdar B«^lm«raal «a<^d.t'«n4 
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Cratmli^a Itft^utftAft' 
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3. MEDIA/INSTRUCTIONAL METHODS DISCUSSED 
3A. COVMUMCATrONME04A 3E VISUAL AIDS 
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□ nrt>t 

□ taoa paca'ii^ 

□ Na^ord riira< 

□ LUtani'ig La« 

□ «iimitr>prSiiant} 

O ^«>>»^ P)viu>a ai n»caiiti>a L««pl! 

□ VaartMo^ 

□ M.c<«'«"n 
Aud<e«iauar 

□ ftMjnd'|f>(*t^'0fa<1I» 

□ Mo1<an P^ctLi^a *» 
Maoai>ib«a Le»a 

JB. ENVIRONMENTAL MtOiA/AlDS 
a J"^oiaiOf 

□ Pa-t frtk »<alna» 
D Wivofa f Ma P'l'na< 
Q >i«<a<}kfa« Tra'hai 
0 Cat^at 

□ na^aPia« 

O ftoofr^raTtJ 

□ OB*'iiio'»ail«iu*'^»v« 

X. TUTORIAL M€0>A 

□ cm c**i 

□ raMMin»Wach,rHM 

□ caa<Ai<^ Lib 

3D. MULTIMEDIA 



Q>i«nka 
Qtrapti 

□ Ofafti'a'^* 

□ Q'ap^a 

Q MKin'i'^fc' 

Ora*^«B» 
0 PHata«>»ik^« 

8Cul A iWavl 

□ Mw)«« 
Praiteiad i^*^»ftH 

□ Ooaoui 
fraja^ttOAa 

□ <Kwhaa4 

r ra#i«r*'*Ac :at 

□ iiwai 

METHODS OF INSTRUCTION 

□ »wta'ia» 

□ Ltetura 

□ ffac)iat^«n 

□ Oi«(wailan 

□ Oanna^itrtMan 

□ fti/nk-iaKan 
Q^aa* ffahlna 
Q taam »ra,ft(rv| 



3G. OTHER: SPtOfY 



TYPES AND METHODS OF RESEARCH EMPLOYED 
(FOR TYPE n DOCUMENTS ONLY) 
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4A. TYPE OF RESEARCH 

Q ftaiK 

Q Aopiiad 
Q r.a(d ftiudT 



4B, METHOD OP RESEARCH 

□ Hmo'icaJ 

□ {>«acriptKa 
a Ca*« A 'Mid 

□ &*rara»«ra^ui 

□ Cauad Ca'nfta'tiiva 
a* ' la r«»t *acta" 

□ CiHfKir^nal 

□ trt*a I^MdrrtaMat 

□ Oua*" e»o*km*rtfai 

□ Att'o'* 

□ OiM», ftpan** 




Figure I. (continued) 
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predented In a technical research memorandum (see Reference 
Number 10) i 

7. Cross Reference * This section of the Abstract Form was used 

to reference the sequence numbers of closely related documents « 

Page 2 of the Abstract Form (^^Second'^ fa^e) 

8. Sequence Number » Same as Number li 

9. Bibliographic Reference > The complete bibliographic reference 
was cited by the abstractor* The American Psychological Asso- 
ciation (APA) Publication Manual provided the standard format 
for citation* Excerpts from the 1967 edition of this manual 

^ detailing the mechanics of citing references are Included in 
the abstractor's guidelines* 

» 

10, Body of Abstract * The body of the abstract was prepared ' 
according to t|ie procedures described in the abstractor's 
guidelines* It consists of a condensation of the document 
content* (The procedures for completing the body of the 
abstract differed ^ depending on whether a Type I or Type 11 
abstract was being prepared*) 

11* Evaluation of Document * The evaluation was prepared accord- 
ing to the procedures in the abstractor's guidelines. 

12. Number of Pages > Number of References > Initials of Abstractor * 
These items i filled in by the abstractor > provided a record of 
the total number of pages and references in the document » and 
identified the abstractor* 

Page 3 of Abstract Form (^'Last'^ Page) . 

13. Sequence Number . Same as Number 1. 

14. Summary and Evaluation Form . The "last" page of the Abstract 
Form served a fourfold purpose: 

a. The abstractor used it as a convenient tool for sum- 
marizing a document by filling out the form while 

^ reading the article. The process of completing the 

form furnished the abstractor with most of the infor- 
mation necessary to prepare a Type I or Tyffe 11 
abstract. 

b. Filling out the Evaluation Checklists portion of the 
form (14A) provided the abstractor with a convenient 
summary for writing the evaluation portion of the 
abstract. 

c. The form provided a useful encapsulation of the type 
of document I topics discussed > types and methods of 
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research employed or discussed, and media and approaches 
to training discussed. A user of the paper technical 
data file can assess In a glance the suitability of a 
document for his needs. 

d. Finally, the most important rharacteristics of the docu- 
ment are synthesized in one place, thus facilitating 
the computerization of the technical data file. 

The Evaluation Checklist (14A) was completed by following ttie 

instructions printed on the form. The ISD topics covered in 
the report were checked off in the column(s) corresponding to 
the type of document being reviewed (14B) , Next» the boxes 
beside all media and methods discussed in the article were 
checked (14C) , Finally, for Type II documents, types and 
methods of research employed were marked (IAD) , 



The Selection and Training o^ Abstract o>r£ 

Selecting Abstractors . Four abstractors were selected, based on 
application of four Selection criteria. Each selected abstractor was 
required to : 

1. Have a degree in Psychology, or an area related to Instruc- 
tional Technology. 

2* Have a general knowledge of the abstracting process. 

3. Be familiar with the instructional system development process. 

A. Be qualified to evaluate experimental studies in terms of their 
design, methodology, and application of statistical techniques. 

Candidates satisfying the selection criteria were given one Air Force 
guidance document (e.g., AFM 50-2 — see Reference Number 2), one experi- 
mental study, and a copy of the abstractor's guidelines • They were asked 
to prepare abstracts for the documents using the guidelines. It was found 
that abstractors who produced acceptable abstracts under these selection 
conditions almost always, with formal training and a short period of rele- 
vant feedback, produced excellent abstracts in the production phase. On 
the other hand, those who produced marginal or poor abstracts during selec- 
tion did not usually exceed marginal performance during production. Once 
the abstractors were selected, they were given reading materials and 
scheduled for a training seminar* 

The Training Seminar . The reading material?: given to the abstractors 
included : 

1. Schumacher, S. P., Swezey, R, W. , Pearlstein, R. B« , & 

Valverde, H. H. Guidelines for abstracting technical liter- 
ature on instructional system development. Dayton, Ohio: 
Air Force Human Resources Laboratory, Wright-Patterson Air 
Force Base, June 1973, in press. AFHRL-TRM-73- 



2. 



Department of the Air Force. Inaimotional system deVetopmnt* 
Washington, D, Author, December 1970» AFM 50-2, 



3. Campbell, T., & Stanley, J. C. Eocpevimentat and quasi- 
expeinniental designs for research. Chicago; Rand McNally, 
1966. 

4. Ten examples of Type I abstracts and ten examples of Type II 
abstracts (a sample of each appears in Appendix I). 

The seminar was devoted to a practlcum in which the following activi- 
ties were accomplished: 

1, The abstractor's guidelines were reviewed and discussed* 

2i Abstractors prepared two Type I and two Type II abstracts from 
original documents* All abstractors reviewed the same docu*- 
ments and participated In a group discussion and critique* 

On-Jo b Trainin g. The abstractor trainir^^ did not end with the semi- 
nar; intense job training was given when the tbstractors began to review 
documents* One experienced staff member was appointed as supervisor of 
the abstracting activities and accomplished the following on-job training/ 
quality assurance activities: 

1. The first document abstracted by each abstractor was indepen- 
dently abstracted by the supervisor. 

2* The two abstracts were compared and appropriate feedback was 
given to the abstractor. 

3. For the first month of abstracting activity, every fifth 
abstract prepared by each abstractor was critiqued by the 
supervisor and appropriate feedback given. 

4. As Judged necessary, later abstracts were spot-checked by the 
supervisor to guard against deterioration of quality. 

« 

The Assignment of Articles to Abstractors 

The assignment process wa^. accomplished in the follow:;\ig five steps: 

J 1. A competency profile was developed for each'^fiSstractor 

(Figure 2), This proille Indicated knowledges possessed by 
the abstractor as well as the degree of difficulty that he 
could handle regarding the general area(s) likely to be 
covered in the articles. 

2* Each article selected was assigned a similar profile based 
on the title, mini-abstract, and source of the document* ' 
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AJIEA 


DEGREE 


OF COMPETENCY 




Great 


Moderate 


Minimum 


Statlstlcd & Experimental Design 






X 


Educational Technology 




X 




Military/Industrial Esoterlca 






X 


Cybernetic Detail 


X 







Figure 2. Sample Abstractor Competency Profile 



3* Profiles were matched In order to assign articles to 

abstractors. For example, an abstractor with a minimum level 
of statistical competency /as In Figure 2) was not assigned 
an article (or a journal) with difficult statistics and experi- 
mental design « 

4, As a further check i the abstractors examined each document , 
and were obliged to : (a) decide If the document seemed too 
difficult for themi and (b) return such documents for assign- 
ment to another abstractor. 

5. The order In which the selected articles were assigned to the 
abstractors was determined by such factors as: 

a. The amount of material of major Import In the article 
as opposed to Its minor detail. 

b. The amount of material In the article with obvious 
military training application. 

c. The amount of contemporary Information In the article 
as opposed to possibly outdated information. 

The Preparation of Acceptable Abstracts 

The details of the abstracting process are presented step-by-step In 
the abstractor's guidelines (Reference Number 9). The process was proce- 
durallzed as much as possible to insure that comprehensive abstracts were 
prepared. To Insure the consistent high quality of the abstracts, three 
quality assurance measures were employed throughout the contract: 

1. The supervisor made a technical evaluation of five percent of 
the abstracts, selected at random (stratified by abstractor 
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work load). To accomplish the technical evaluation, tho 
supervisor: 

a. Reviewed the source document and described the criti- 
cal contents which every abstract must contain. 

b. Compared the abstract against thfi list of critical 
contents* 

c* Provided feedback to the abstractor when the abstract 
was def icient/^or excessive, and had the abstractor 
revise the abstract accordingly. 

d. Evaluated the revision and provided feedback, as 
needed • 

e» Reviewed enough other abstracts (from those prepared 
previously) to insure that the problems were not per- 
vasive. 

f. Kept a record describing each abstractor's performance 
in the technical evaluation. 

g« Initiated remedial training or other appropriate action, 
to Insure high quality of abstracts. 

2. Five percent of the articles that were abstracted at libraries 
were cross-assigned to two abstractors. The supervisor com- 
pared these abstracts to insure that the guidelines were being 
conscientiously applied when the abstractors worked on their 
own in the libraries, 

3. A random sample of approximately 25 percent of the abstracts 
was edited by the supervisor. The editing process Involved 
reading the abstracts to Insure they were complete, clear, and 
internally consistent. A vocabulary check was also conducted 
to Insure that the selected search terms accurately repre- 
sented the detailed written abstract. Significant editorial 
or vocabulary problems were noted on the abstractor's perform- 
ance record and remedial action was taken, as appropriate. 



The File Development 

Obviously, the most basic requirement for the file was that it be 
readily usable. Therefore, considerable thought was given to the nature 
of the file and its indexing system. The basic requirements determined 
for the technical data file were that It be in paper form, that it be 
Indexed and cross-indexed, that it be amenable to eventual translation 
into a computer format, and that it be updatable (living). 
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Th e Paper File 



A number of ir.dexing approaches were considered, but the system which 
best satisfied the requirement for a paper file appeared to be a form of 
coordinate indexing (Reference Numbers 1 and 12)* The coordinate indexing 
system employed involves the use of a Master List and coordinate Index forms 
(index cards). Th<^ Master Llat contained the titles of all abstracted 
documents numbered in the order in which they were abstracted, (See Refer- 
ence Number 8 for a complete bibliography.) Each Index card bore a head- 
ing consistitig of one general search (retrieval) term (see Reference Number 
8 for complete list of retrieval terms) , and each card was Itvided into ten 
columns. After a document was abstracted, it received the next sequential 
number on the Master List, That number was then entered in a column on 
each of the coordinate index cards headed by a retrieval term that might be 
used to locata the document* If the last digit of the number was 0, for 
exdmple, the number was entered in the first column. If it was 1, it was 
entered In the second column, and so on. Having ten columns on each card 
facilitates cross-comparison between two or more retrieval terms because 
it insures relatively short lists of numbers in each column. Figure 3 
depicts an example of application of the system for two retrieval terms — 
videotape and lecture. Note that document numbers 26, 135, 232, 311, and 
413 have been circled. This indicates that the documents (and abstracts) 
bearing the numbers would contain material suggested by both videotape and 
lecture. 

The coordinate index form is a versatile temporary storage medium 
which facilitates the search of multiple key words in a paper file. It 
also permits easy translation to a variety of other systems. Of course, 
the computerization of the system will permit an even more efficient means 
of retrieval, and cross-sorting by multiple variables. 



The Computer File 

The considerations for easy computerization of the file were more 
complex. A primary consideration was to permit flexibility in the coding 
of the document information. Also, some efficient means of inputting the 
paper file to the computer was required. These requirements had to be 
satisfied (in so far as possible) without anticipating any specific com- 
puter search-and-retrleval system. (The draft report of a feasibility 
study for the computer search-and-retrleval system was completed In March 
1973 — Reference Number 6.) 

Design to Permit Ease of Coding . Flexibility was the goal. Without 
prior knowledge of the design requirements for a computerized file, It was 
decided to provide the designer with as many coding options as possible. 
The following 14 information items were selected as potentially useful: 

1. Specific bibliographic information (e.g., author, date). 

2. General search terms (a defined list of terms to be used by 
the abstractors in coding the documents). 
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Figure 3. Sample Coordinate Indexing Application 
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3. Unique search terms (Important terms not listed as general 
search terms). 

A. Needs for additional research, 

5. Cross reference to related documents. 

6. Complete bibliographic reference. 

7. Abstracts divided into sections (for Type I: Contents and 
Clarity; for Type II: Purpose, Methods, Results, and Dis- 
cussion) . 

8. Supportable conclusions (distinguished from discussion of 
results). 

9. A summary evaluation of the document. 

10. Individual evaluation ratings covering major aspects of the 
content, writing, methodology, and design of the studies. 

11. Specification of the type ot article (e.g., opinion article, 
correlational research) . 

12. Specification of the major ISD activity to which the work 
relates. 

13. Identification of all media or instructional methods dis- 
cussed or investigated at Length in the article. 

lA. Identification of the type and method of research (e.g., 
field study, quasi-experimental). 

To facilitate coding, each of these items was separately entered on 
the abstract form. (See discussion on Pages 19-24.) 

The feasibility study (Reference Number 6) demonstrated the use of 
computerized search-and-retrieval system. The demonstration system 
required four additional information^^ items that were not specifically 
covered on the abstract form. The^fe items, which were easily derived 
from the Type II abstracts, areuj 

1. Subject population description 

2. Independent variable (s) 

3. Dependent variable(s) 

A. Measurement technique(8) /statistical method(s) 

Design to Permit Ease of Inputting the Paper File to the Computer . 
Initial estimates were that the abstracts would require over 3,000 singl 
spaced typed pages. The desirability of avoiding retyping these pages f 



computer Input (e.g», by keypunch) is obvious. Three options for providing 
a more dlroct means of inputting were considered: 



1» Typing in Optical Character Reader (OCR) font. The typed 
abstracts would be processed through an OCR and translated 
to computer-readable tape. 

2. Typing in OCR font on an IBM Magnetic Tape Selectric Type- 
writer (MT/ST) • Copy would be processed as in No. 1 above. 

3. Typing in standard font on an MT/ST and converting the MT/ST 
cartridges directly to computer readable tape* 

A survey was made of the costs and benefits of these options.^ F^ictors 
considered weve: 

1. Availability of conversion equipment (OCR or MT/ST cartridge 
converter) tb the Air Force Human Resources Laboratory ^ 
(AFHRL), 

2. Costs to AFHRL to use conversion equipment. 

3. Comparative typing-time requirements* 

4. Readability of the paper file (e.g., currently available 
OCRs require all capital letters and special characters to 
dentate specific functions). 

5« Ease of making corrections » additionsi or deletions to 
abstracts prior to conversion^ 

6» Base of entering additional codes prior to conversion. 

Following a study of these considerations » the third option proved to 
be clearly optimal* Consequently! it was recommended that: 

1. The file should be typed In a standard font for ease of 
readability of the paper file. 

2. The body of the abstracts should be typed on MT/ST tape 
cartridges. « 

3. The "first" and "last" pages* containing the majority of 
the accessing information for each abstract, should no t be 
typed on the MT/ST until after the appropriate access codes 
were specified. 



MT/ST tapes provided a convenient means for adding access information after 
the feasibility study Identified the codes. Ten blank lines were left at 
the end of each MT/ST tape to permit later addition of access codes within 
the body of the abstract. 
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4. The IBM 2^95 (available to AFHRL) or similar conversion 
equipment shoCild be used to convert MT/ST tapes directly 
to computer-readable tape»^ 

Tryout of the Planned Computer File . Two tryouts of the computer 
filing 'system were conducted* In the first, a sample of 10 MT/ST cartridges 
were processed with the IBM 2495 at the Foreign Technology Division at 
Wright-Patterson Air Force Base. This trial successfully demonstrated that 
the MT/ST cartridges could be translated to computer-readable tapes. The 
second trial was conducted by the University of Dayton on a sample of 500 
abstracts. This trial successfully demonstrated the feasibility of the 
storage-and-retrieval system the University of Dayton was evaluating. Both 
trials indicated several important considerations for preparation of the 
MT/ST tapes: 

1. The alignment of the MT/ST characters on the MT/ST tape 
bands is critical. (The MT/ST reads the information rela- 
tively slowly and accepts a wider alignment tolerance than 
the faster IBM 2495 converter.) 

2. Additional paragraph and other information delimiters 
should be added to the MT/ST tapes, (This could be handled 
by revising the search-and-retrleval system program.) 

3. The sequence number should be entered with each separate 
file* (This could be handled by revising the search-and- 
retrieval program.) 

4. Lower case "L" and alphabetic **0" should not be used to 
represent the numbers one and zero. (This could be handled 
by revising the search-and-retrleval program.) 

5. Terminology should be as consistent as possible within and 
across tapes (e.g., AV Communication Review, not A V Commu- 
nication Review or A-V Communication Review, even though 
the publisher used both of these in the past) • 

These findings represented relatively minor problems. The MT/ST tapes 
are suitable for entry to the computer. With minor refinements, search- 
and-retrieval programs can be adapted to handle the present MT/ST tape con- 
figuration. If an operational final version of the search-and-retrieval 
system is prepared, close coordination will be necessary to insure that 
additional MT/ST tapes satisfy the program input requirements. 



After the codes were determined in the feasibility study, a second com- 
puter tape could be made from appropriately coded MT/ST tapes of the 
Information on the ^'flrst" and "last" pages. This information could be 
merged with the primary computer tape containing the abstracts. 
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Summary of File Contents 



The final technical data file consists of the following s 

1. A coordinate Index of the general search terms » 

2« A complete card Index of all titles searched i Including 
(when available) the short abstract from the document 
source (e.g., Psychological Abstracts ). 

3. A complete paper file of abstracts. Each abstract la in 
three sections: 

a. The "first" page contains a separate listing of 
information items (bibliographic information, search 
terms, and needs for additional research). The 
separate listing of these items permits easy coding 
for computerizing the file. 

b. The "second" page contains the complete bibliographic 
reference, a Type I or II abstract, and an evaluation 
of the document. This portion of the file was typed 
on MT/ST cartridges which can be easily converted to 
computer-readable tapes. 

c« The "third" page contains a completed Summary and 
Evaluation Form for the abstract. This form identi- 
fies numerous parameters related to the content, struc** 
ture, and quality of the article. The information can 
bo easily coded for computerizing the file. 

4. A complete bibliography of all abstracted documents on MT/ST 
tape. The bibliography, ordered by sequence number^ and a 
complete key word index is presented in Reference Number 8. 



The Production of the File 



File production was divided into two general activities; (1) prepa- 
ration of the abstract from the original document, and (2) preparation of 
the final-typed abstract on MT/ST tapes. 



Producing Abstracts 

The abstract production process was discussed earlier (see Section II, 
The Abstracting Process). The three key elements of the process were! 

1. Careful selection of articles for abstracting based on a 
prescribed routine. 

O 
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2i Assignment of attlcles to abstractors on the basis of a 
profile of abstractor competencies* 



3. Extensive quality control measures by the supervisor to 
ensure that abstractors were following the procedures In 
the abstractor's guidelines* 

Including supervision time and the time to hire and train abstractors^, 
approximately 2-1/4 hours were required to select the article and produce 
an abstract that was ready for typing* This represents an average time. 
There were significant variations, depending on whether ^ Type I or Type II 
abstract was being produced. 



Typing Abstracts 

The process of typing abstracts on the MT/ST required specific proce- 
dures to insure that they could be processed on the IBM 2495. However, in 
order to produce a complete file, "first*' and "last" pages had to be pre- 
pared as well* 

" First" and "Last" Pe^es . As abstracts were received by the clerical 
staff, the following steps were taken to produce the file: 

1. The bibliographic reference was typed on the Master List (on 
the MT/ST). The reference was typed in the standard format 
prescribed in the APA Publications Manual. 

2. Each reference was bntered on the card file. 

3. The general search terms on the "first" page were checked, 
against the list of standard terms. As necessary, changes 
were made to insure that the listing on the abstract was 
accurate, and that the terms were entered in the coordinate 
index file. 

4. The "first" page was typed and proofed. 

5* The "last" page was produced and proofed. 

Typing Abstracts on the MT/ST . To successfully prepare MT/ST tapes, 
a number of special typing considerations must be observed. Among these 
are: 

1. Preparing the tape. 

2. Coding the data fields on the tapes. 



The time to hire and train abstractors proved to be a significant factor 
since due to attrition, 13 abstractors were employed In the course of the 
effort. 
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3. General typing of the abstracts » 
4t Entry of special characters/syttbole* 

5« Administrative procedures to keep the production process on 
schedule. 

The details of the MT/ST typing considerations were suimarlzed in a project 
memorandum. This memorandum is attached as Appendix IZ of this report* 



35 



SECTION III - RESULTS AND DISCUSSION 



Overview 



The final file comprises 2>693 abstracts of technical literature, and 
Is the most comprehensive file produced to date in this area* Schef fieri 
et al, (Reference Number 6) , summarize the characteristics of the file: 
^\ * 4 the comprehensive abstracts represent a thorough document surrogate 
data base from which one actually can retrieve facts and principles as well 
as evaluate data regarding ISD« As such| the data base is unique^ since 
other IS&R (information search and retrieval) systems in no way provide the 
depth of analysis and comprehensiveness of coverage in the field of train- 
ing and educational research as well as instructional system design**' 



In this section the sources searched and the compendium of research 
and development needs are reviewed • This section concludes with recom- 
mendations for completing the file. 



Two types of sources were searched: general information sources and 
specific Journals* 



General Information Sources 

The general information sources in Table 1 (p» 14) were searched early 
in the project* The focus of this search was to find significant texts and 
methodological development articles^ bibliographies » and reviews which had 
a significant Impact on the emerging technology* Most of the first 700 
items in the bibliography were identified from the general sources (Refer- 
ence Number 8). Approximately 20 percent of the first 700 items resulted 
from the references in texts » articles, bibliographies, and reviews that 
were abstracted* The general sources also provided a resource for iden- 
tifying journals which might be profitably reviewed. As mentioned in 
Section II (p* 13), the search of specific journals offered significant 
advantages for abstractor assignment and for control of the search* Fur- 
thermore, although it could reasonably be assumed that much of the dated, 
but useful experimental data would be incorporated topically in texts and 
other Type I documents, the journals themselves would be the only adequate 
source for preparing the informative Type II abstracts. 



m 



Sources Searched 




Specific Journals 



Forty-eight journals were reviewed. As mentioned earlier, the strat- 
egy was to review the most recent volumes first and then work backwards. 
Journals which yielded the highest number of relevant articles were 
Searched more completely (in some cases as far back as 1952). However, 
twenty-one of these journals provided few relevant articles. When upon 
careful review of a number of issues, a journal was found to be generally 
not relevant for ISD purposes, it was not searched any further. (This is 
not to say that these Journals would not provide information useful for 
ISD if they were searched with broader selection criteria.) 

Figure 4 shows all Journals searched in which relevant articles were 
found. Figure 5 shows the journals which provided three or fewer articles 
satisfying the selection criteria. Appendix III lists the references for 
articles that were selected for abstract. Ing^ but which due to time con- 
straints, could not be abstracted. 



The Compendium of Research and Development Needs 

Many authors, especially those of Type II documents, cited needs for 
additional research. Each need was recorded on the "first" page of the 
abstract form. These needs were arranged In a compendium of research and 
development requirements and submitted as a separate report (see Reference 
Number 10). 

Many of the requirements in the compendium can be understood without 
reference to the original document. However, before utilizing specific 
items in the compendium, the user will probably want to examine the orig- 
inal article, its title, or the abstract. For this reason, each need In 
the compendium is concluded with a reforence to the sequence number in the 
file and the year of publication of the original article. 

The contents of the compendiun are summarized in Figure 6. Some of 
the more broadly stated needs appear under two ov more categories in the 
compendium. 



Recommendations 



The technical data file developed under this contract covers a con- 
siderable portion, but by no means all of the relevant technical litera- 
ture. A complete file would contain from 10,000 to 15,000 such abstracts. 
Because of its bulk and the difficulty of distribuf;ing the paper file, the 
utility of the paper file is severely limited. Therefore, early develop- 
ment of a computerized file is highly recommended. 
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DETERMINE JOB PERFOFMANCE REQUIREMENTS 




Task Descriptions 

Performance Requirements 

Classification of Tasks 

Predicting Future Performance Requirements 




DETERMINE TRAINING REQUIREMENTS 




DETERMINE CRITERION OBJECTIVES 




DEVELOP CRITERION-REFERENCED TESTS 




Tast Construction 

Effects of Testing 

Test Reliability and Validity 


• 


SELECT MEDIA/METHODS 




Audio Devices ^ ' 
Visual Devices 
Audio-Visual^ Devices 

Visual Aids ' 
CAI , 
Programmed Instruction 
Instructional Devices 
Methods of Instruction 
Media Comparison 




DEVELOP INSTRUCTIONAL MATERIALS 




Learning Theories 
Instructional Techniques 
Factors Affecting Learning 




EVALUATE INSTRUCTIONAL PROGRAM 




Analyze/Evaluate Program Data 

Evaluate Instructors 

Investigate Student Characteristics 

Investigate Effects of Student-Instructor Interactions 
Investigate the Effects of the Learning Environment 






Figure 6* Contents of Compendiujn of 

Research and Developnjient Needs 
(See Technical Research Memorandimi 
Reference Number 10) 
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Cotaputerlzlna the File 

The feasibility of computerizing the file has been demonstrated* The 
primary recommendations for its development are: 

1. The final requirements for abstract content and coding in 
the preparation of the abstracts should be developed Jointly 
by the organizations responsible for ccmputerizing the file 
and for preparing the abstracts. 

2» Provision should be made in the software of the search-and- 
retrieval system to accommodate the abstracts produced to 
date. Possibly some level of intermediate processing will 
be required after the computer tape of abstracts is produced. 
Additional abstracts that are prepared should not require 
intermediate processing. 



Completing the File 

The recommended procedure for completing the preparation of abstracts 
for the file is: 

1. Determine and revise (as necessary) the abstract/production 
process to insure that abstracts and MT/ST tapes are compat- 
ible with the requirements of the search-and*-retrieval system* 

2. Prepare abstracts for those titles already selected (see 
Appendix III) . 

3» Complete the search of journals listed in Figure ^ back to 
1952. 

4. Return to the g( .eral information sources listed in Table 1 
and search thes sources for experimental (Type II) studies 
not already in :c\e file. 

5. Conduct a comprehensive search, and abstract the relevant 
works In the various Air Force Libraries (e<g., the AFHRL 
Library, the Air Training Command Library). 

6. Continue abstracting relevant journals (Figure 4) for 1973 
and beyond • 

7. Develop a program for keeping the file up to date* Minimally, 
this will involve continuation with 5 and 6 above, as well 

as the identification of new general sources and texts rele- 
vant to the technology. 
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Suiting the File to User Needs 



To be maximally useful » the search^and-retr level ays ten and the 
abatracta muat be aulted to user needa» It la auggeated that the fUei aa 
now configured (2|693 abatracta)» be aubjected to uae*-te8tlng In both the 
paper and the computerized aearch-and-retr level conf Iguratlona, Caytalnly 
the file, In Ita current atate» will aid practltlonera and reaearchera* 
But aa It la uaed, waya to Improve the abatracta and the aearWand^retrleval 
mechanism will aurely be determined* The ayatem aho.ild be connWlly 
reaponalve to uaer needa. If the production ayatem la modlfled^/Uatlsfy 
theae needa, the addition of further Incrementa of abatracta vlll^pj>rove 
not only the comprehenalveneaa of the file, but alao Ita utility. 
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EXAMPLES OF A TYPE I AND II ABSTRACT 
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SEQUtENCe NO.. 



ABSTftACTTYPE: 1 

A&STRACT FORM 

AUTHOR(S): G, L. Cropper 

TITLE: The Design of Stimulus Materials In Response-Oriented Programs 



PUBLISHER: AV Communication Review 

REPORT NUMBER: Volume 18 DATE: 1970 



GENERAL SEARCH TERMS: 

Auto-lnstructlon Program 

Chaining 

Cue 

Discrimination 
Generalization 
Graphics 
Handbook 

Learning Facilitation 
Materials I Preparation 
Participation of Learner 



Practice 

i^jcesentatlon Methods 

Printed Material 

Programmed Instruction (PI) 

Prompt 

Response 

Sequencing 

Stimulus Characteristics 
Stimulus Presentation 
Time Factors 
Visual Materials 



UNIQUE SEARCH TERMS: 
Response-Oriented Program 
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000691 600 Cropper I G* L. The design of stltnulus materials In responses- 
oriented programa* ^^AV Coinaunlcatlon Review," 1970, "18, 129-159« 

000691 700 This paper proposed and offered evidence that* the impact of 
response practice on learning effectiveness and efficiency can be heightened 
by systematic organization of stimulus materials that control such practlc^« 
A discussion of the similarities and differences between response- and 
stlmulus-qriented technologists was presented* One of the differences is 
in the systematic approach which the response-oriented technologist takes in 
organizing stimulus materials so that the practice of relevant discriminations, 
generalizations, and chains may be optimally facilitated! It is a question of 
presenting the kinds of contexts and cues that can contribute optimally to 
the acquisition of those three skills that underlie all learning) l«e., 
discriminations I generalizations, and chains* 

Spatial organization of stimulus materials has the potential for diverse 
applications. The applications described in this report included: 
(1) language learning; (2) learning facts, concepts, and principles in 
physical science, behavior theory^ and instructional technology; and 
(3) learning procedures « The effectiveness of the techniques depended on 
two properties of diagrammatic presentatlon-^spatlal organization and 
the capacity to present the "big picture*" Spatial variables included 
adjacency, separation^ grouping, enclosure, and order* Comparison of 
spatially enclosed and separated stimulus material formed the basis for 
the practice and acquisition of discriminations and generalizations* 
Relaxing stimulus material in one square to that in other squares formed 
the basis for the practice and acquisition of associations and chains* 
Practice and acquisition of discriminations, generalizations, and chains 
was further facilitated by "big picture" statements presented in a diagram 
appearing on a single page* 

This document included several examples of stimulus materials organized in 
the ways described. 

000691 800 EVALUATION— This article was of very good quality; the 
exemplary diagrams which were Included were extremely helpful in understanding 
the text. Examples of applied research in this area were discussed in 
detail. Applications relevant to ISD were presented which seemed to be 
particularly related to the development of handbooks or manuals* (R«ll, 
P-31, EMW) 
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i CHICKLI$TSFOAtVAlUAtlNQ 
ryi^E I fit TYPfe II OOCUMENTS 

1A. INSTRUCTIONS 

Lilt Of IC. All lti<r| m»v f\6t Ki^y (4 | ip«c^*k ilOCviTtii^r 

# If lunt ^ai not Wpl* m my ihty. tht NA in iMti t» 
lh4 rtfl «! It4.n, 

• Hi rfOCuiti#rtt fully IttViM m 4t*m trlt«nen, mtfk \h» \ b«« |h| 

♦ It « document «nly Paftr*llv ••tItfiM »n lt«m ut^lon. m«,b ttsi 
} <o tH*l«ft al iM tt«m. 

# If • doc\jm4A( 'tJii t« ««tf«fy ift lt»m erittf^flrt. /n«<k 1 b«« 10 tM 

ifiiisisa. ■ 

1«. EVALUATION CHECKLIST FOR TYPE I ABSTRACTS 
Dt9Cflptiv« Sumfn»rv ol CoMeot: 



tf OUthfCI NO. 



1. TItTi Ot dr>cum*nt tfMCrito*! <ont»«it4 

«. n«a*0«tl (or d«»llnf «V||K tM» irM fri tlMr. 

1 P'O^^tm IrM IicOmatM AdMiuattVy. 

i. ApproKh to erObFtm trta |« *ppropr(it«k 

», ft»»«tl-Of »hip of Ih^l dMurt^tOt to Oth«f rtltvani qi»r»tut» U llHld. 

10. ^9vi»<^% tif r#M*'th CiMr Md CO^IM. 

11, «*f(«Wf of itMi'th «pOCi»v dM'gni i^npipy^d, 
If. OttphI ind llblM irft priHfittd ciMrly 

11. CerciuiiOrti «t»tiJ ci«i)rry. 

>4. CofHiuHom *r» kjMTint itM by d«ric» prM«fs|M In thi docurtttnl 

14. CoAc»ul)«n( «<« co1^rt«d (0 c>t» pfo6l«m irM iHKwi»*d tn documtnt 

14. «iy.plle jiloni for f wN'l 'tiMrch tri IcgktMy fo«mLji«t»d. 
I a>bl>Ofl«»Dhy 1« pfofnttyf Orgsnff M irtd DrH«<tt»d clWIy. 

Clirity «nd U»«fu(ncss: 

15. Oo«Mm«nt do«i not ttt^M^ *t'»bti«h«d Of InetptM, 
1>. Oocurr>«M p^»t«<^r» »pp)lc*()o/>« r»f#viM to iSO. 
70. Olfteurritnt p^M'^ti. roix«t>l« f»l«vant to ISO. 
?l. CorHiul»oni •<« intpprt^nt Ipr iSD. 
21 Stv>« ot Mm|4r9 i| »f>otoct>»\» tot tyolc»i ISD uM^i. 

34. Do<u<rv««it ;t r**Mn.*bly «««y to foUoM •n<l la rsoi confuting 
J4. To«« of lh» dPcurr^Al diioUvt onbi|»#vJ, lmp«^ti«i •eltrttlfk »n^d«. 

10. EVALUATION CHCCKLlST F0« TYPE II ABSTRACTS 



1. Tktri of dixgmani adMulifly dMcribH eortt«nlfc 
2< Prob>*m l« mtM ciM'ly. 

4. AtStirrptloAl «f* »tl1»dC>*ir^y. 
*. Cimtt*t<ani of tK« »tudV »r» r*ttd. 
ft, ImporMnt tartni $tt atUftti. 

J. Av'iiionlhip gf iha tftwbiwm iw >J»»^1mv* i*«**ft.h 1| ttWf. 
». PrtviO^ rllMrcb U'«vl»A««d •d*<«<jMlly for •uthor'l purOOM«> 
9. Hyooiti*^ »f| ttat*<l elairly. 

>op«ii*tl«A if ttMouttfiy dMertb«d M lo 111*, loctllD'V »««. iftd ot1>« 
ch4r«ct»fU1kl r»l«vaitl to lh» r•«•irc^. 
i 1 I I I *• MmN*^ *f MT^pMri* (| Miquittly d«*<rlbOd ind i* ippi op^lctli 
^ i I . ] Sf'T'P'* "»«4»Q«»try dMArlbtd at lo lo<»tlon. »nd 0*»«» 
^•fAvOnt cHpf^Cftrfttfck. 
1). I^««««rch dM<«n d*acr>t^ fulty, 

14, ft*tlOA*i*ra^ tpAcl'k rnMreh dHlg'% la a^aouataiy |iiatlf|»d. 
f ft, l'lM««rch dMi|r^ =1 b#at for ao>vTiOn ef Hit WftbJtm, 
14. tfi* aiffAcH of lrt4«ol*v«i»r^t *arr»(»rM art not eo^rouf^d*tf *»?th it>« 
- . mtltLtt of u(icontro» *d, anlrkntoua varTabiM. 
L t 1 1 i 1'. Th* •itMcii of Wap*n«nt v»ri»bt»i at> not eoAfpurtdod yylth tha 
tW^cfi of mot^vailonal teirfibra*. 

[ r 1 It Control ««ouM ira app^eCMit* »or cpmp«rtton «kO«rln%«Ata1 

I», lid^o^^'dant varlab'at an Kioqua^fy lP«cifl*<t. (QwaMtatlvaiy) 
20, D*t>««^arvt »ariabtai «ri adaq^Jal•:y io«c«(!*d. (CkiamitSrCiv) 
7\. VoriabtH ptM opOf atlO/ijfiy OvfSriid. lOutMltatlv*, 9«ri«raMv> 
72. Vpp>op'l«t« rr>Maur»i ira a«r«cta<f fo* d*p«nd*ol vflAbPn. 
n. C«parlm«fM*r marvlpw'itM wt^at h« uva ^« rnanlpulattl. 
2'4. CondJflorta uAdar l*f^1e^ a» par f fTv»r.t «wm «o/%d«iad *<»cUotly voc^fFtd 

34. ^t«•a<e^ d»il«'% fa f^>« of olhar looc^fk ^taknaaa^*. 
2*- Oaia-yalKarlog rh«iHDd« br procaduita ar* d««CrIb*(S lA tufflctOfil d*t4ll 
»o allow r#p»rai>on. 

[ ~1 1 11 3^ D•r•■B*t^•'l'^» m*thod< Or prpca^ra* rollOvM from itX rMMrcK dHtfft. 

moihoot Of P»ocadura» a»i aF^O^aprUia to iha io>ut1on b( |Ha prob^»m. 
I l l I ~] 0at4-9»thari»*i maiho<Hor pfoc»dur*t do r»oi Iniroduc* irti<aci\ja( braaM 

- {i»to d«i>%, 

QIZHIZD ^ «»*MrcK«i hat UV*r\ appropnata at**» to ailmMata o* control f« 
»Hp«r('Tva«^l»» bia« and Mlf fu'f^llirf p#aph*cy. 

Rtsv^lss. 

30. Oil*^»ath»rlA9 (T%«tho<fi at ofOC«dura« ark u«*d<etr«ctry. 

n*l1*bl'ifv of rri 4* n..<ii>fl (Mtfwmanli n tatablta^. 
37. Validity pf r^Mtu ring I rnrunnofit* »t aaiabUahad. 
33l App«opr|»ia mathoda art wtpctAd te *ntty*k rS*OaU M «pcord•^Ca 

M>(th >hPM if>dic*t9^ br t>« r#a«*fcH d**)*^ 
M, M*tNod« ui«d 10 arkftlyta iM data ara Laod corrKtty. 
34. A*«u1tt Qf |K« an*lv«*tafa pf«Mr^t»d c<Mrly. 
3*. OrapN* and t-kfaPM *r« p^tMntftd cCM'ry. 
3'> Onry a^o^lf leant ranutri art trt>»<tue*d a« rf>a«Mrf(i,( dat-c 
3ft. Confkrfahca iPwaH ara iliia^<.iMrly and II un^iua^. jwatifiaid. 

Unulual at«ilatka1 n«»(arturM pv urHKunl pppllcattoni of Ittndafd 
oro^Aduiai art rfoaiftbad daaMy and luatlffad. 
da. rran^fo'Mad «iaia «ra eTa«Hy ae«cifi«r( aa auch *a4 Mtif»*di p* aonvtraToi^ 

ar» ttaiad eiaa>i« an«4 adauiia^tiy i jaii'iM. 
DilCuMiOO: 

41. Co«Ktu«toAI • tl«1*d cl««rly, 

42. Cof^c^^ttoAi flub«TaAiHat»d by the wl^a^Mt pr»«aniid. 

43. Gpnaralittrioi^i pra <onfih»d to tK« poouUt^pn trpm whVh tha itmola 
WM tfra^w. 

44 Conciuaio^a rt<ata ofly t« t^a ftpraif>ai«r« of th« ImPatifM^ tfwt 
tMra K>«clfiPd uii^a* aYMflmonial «oMltJOft«. 
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49, infart<^tj«J tna fftt«<orotfv« eoncH/|}oA« 4^4 c^iy ipdclf /«d fi aic^ 

44. Inf4r«ni«ai ind [ni«rpf«t^4 conchii^OAi »ra (uttlf i«d. 

4>. 5p4euinlont. •Maaa4». «ro th*grUin| 41 4 not P»t»«M4d h Wncfwt^ooi 

■■ ■ . <h4i fpdoi* from tho dati, 

^ I I in 4«, ImplicitiOfM for fufwra r4«44r«h »r* lOf kaPly torm^latod. 

Clarity ir^UMMfiH$: 

^4port i» *titt4n <l*4f>y. 
■ft ^if>ort!|lo#ClllvOr^n|iKl 

J*. Acport i| «44«on«bty iOiV to fol^M »ftd k not ton»yalA|. 
■J. ToAoof ih4 r4Wfl d^tptiyi untlpMd. impont*! itPtrtiiik •Mltud4. 

Aapori la tf »ood 4«4rait quality. 
^ Aapo^ la of tf<nportanc» to (hi (>rob^am arp4 
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3. MEDIA/INSTRUCTIONAL METHODS OtSCUSSeO 

3A COMMUNICATION MEOtA 3E. VISUAL AJOS 



Audio; 

D B«*l» 

□ T8P4 PicflJdtf 

□ AKMd Pllyfer 
C Ultamlnfl Lab 

VLauaP: 

□ f l<m»tf3|! ^Stlortil 

□ Votlo<^ Pkturi M A«pai)lK(a Loof) fSHiotl 
^TaaTtooli ' 

G WieiOIOrtTV 
A^dlovliuhl: 

□ FllfTNttTlp/SPuod 

□ Ttl«vli1on 

□ MetlOA ficiurt 

□ MotlOA fktuf* at 
RaotlltJva Loop 

36. ENVtRONME^ALMEDtA/AlDS 

BSI-rTHjtafO* 
PmtkriMk Tralnor 

□ M^bloTMk Tralf^a^ 
G l*'rO<«dur*l Tralnar 

□ Camai 
O f^o'aPiay 
O AoclOdra-np 
G Oo4ri.t^o'uil CqtilpmaAt 



3C. TUTORfAL MEDIA 

□ CAI/CWI 

m frpyammod T»«t 
Q T*achrr%i MpcMrkt« 

□ LHtnlf^ Lab 

3D. MULTIMEDIA 



OrloHkl' 

Q C'w'n 

□ flrapha 
Q iiHiat^atiofta^ 

Q l^oto^rap^t 
txMbIt*: 

aCvt-A-Wayt 
l^liwM 
C Modtia 
PrO(*et«d rina^r 

□ OOPOs^l 
PrO]Mt>Ona 

□ Ovarhood 
T»a« rot 

a SHd«a 

3F. METHODS OF INSTRUCTION 

□ Tutorial 

□ L*ctura 
Q- n«ell«tlon 
Q t>\tt uw/on 

Q D«mo<H|ratro<^ 
O S#tl Inur^»cfiOf> 

□ S'mutatlo/t 

□ fMf Training 
O T«pm rra(Ahi| 
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4B. METHOD OF RESEARCH 

□ HIttortcat 
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O Cm* ^ ^^Id 
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□ Actions 
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000715 600 Dwyer, F, M, Color as an Instructional variable* *^AV Communication 
Review/* 1971, "19" 399-416. 



000715 700 Conflicting research findings were cited concerning the effects of ' 
color in instructional materials. In addition to pointing out this need for 
further empirical study, the author also presented a theoretical justlf Icairion 
for his study, based on the "realism theories" of Morris, Carpenter, Gibson, 
and Dale. 

The study tested the following null hypotheses J (1) there are no differences 
in achievement (on the five crlterial measures specified in this study) among 
students receiving oral Instruction complemei\ted by visual illustrations 
possessing different amounts of realistic detail; and (2) there are no differenced 
in achievement on the five crlterial measures between students receiving black- 
and-white illustrations and those receiving colored illustrStlorls to complement 
oral instruction. 

The independent variable was the type of visual illustrations each group rciceived* 
There were eight experimental sroups and one control groups (1) no vlsualf; 

(control group) J (2) simple line illustrations (black and white)} (3) simple 
line illustrations (colored); (4) detailed, shaded drawings (black and white); 

(5) detailed shaded drawings (colored); (6) heart model photographs (black and 
white); (7) heart model photographs (colored); (8) realistic heart photographs 

(black and white); and (9) realistic heart photographs (colored). (Author 

included samples of Illustrations.) 

The dependent measure consisted of five crlterial measures: Test A—drawing, 
18 items; Test B — identification* 20 items; Test C— terminology , 20 items; 
Test D — comprehension, 20 items; and Test E — total crlterial, which was a 
78-point composite of the first four tetts. The corresponding Kuder-Richardson 
Formula 20 reliabilities were .76, ,86, ,77, ,82, and ,93, respectively. 

Method: The Ss were 261 college students enrolled in a lower level psychology 
course and randomly assigned to one of the nine treatment groups. The Otis 
Quick-Scoring Mental Ability Test was used as a pre-test. The Hartley test 
for homogeneity of variance and an ANOVA were conducted on the IQ scores. In 
no case did the observed values reach the crlterial value for a p ♦LT. .05; 
i.e., no differences among average group IQ's were demonstrated. 

The oral instruction, a 2000-word instructional unit describing the human heart 
and its functioning, was presented to the students by means of an audiotape 
recorder. Audio signals on the tape cued the changing of the slides so that 
both the oral and visual instruction presented relatively the same information 
simultaneously. The rate and number of words per minute were identical for 
each presentation. 

An ANOVA was conducted on the scores achieved by the students in the nine 
treatment groups for each crlterial measure. Where significant F-ratios 
(p ,LT, .05) were found to exist, differences between pairs of means were 
analyzed via Tukey's W-Procedure, 

Results: On ail nriterial measures, significant differences were found to 
exist among the means of the nine treatment groups. Except for test C (F«2.39; 
df«8/252; p .Lj:, ,05), the F-ratios for the ANOVA were all significant beycnd 
Q 1 .01 level. The statistical findings for tests A, B, D, and E wereJ F«8.26, 
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df«8/252, p •LT, •Ol; df-8/252, p .LT. .Olj df-8/252, p .Vt. •01} 

and F-5,92» df-8/252, p .LT* •01, respectively. 

The effectiveness of the various slide sequences In Improving student achievement 
on the crlterial measures was measured via the Tukey U**Procedur6« 

For test treatment 2 was most effective; treatment 3 was more effective 
than treatments 1, 7, 8, and 9; treatment 5 was more effective than treatments 
li 8, and 9} treatti^k)t 7 was more effective than treatments 1 and 6. 



For test Bt treatment 5 was most effective; treatment 5 more effective than 1, 
8, and 9; 3 more effective than 8 and 9; 7 more effective than 8 and 9« 

For test C: treatment 1 was as effective as visual treatments « 

For test D: treatment 1 was as effective as visual treatments; treatment 3 
more effective than 6 and 9* 

For test E: treatment 3 most effective; 3 more effective than 1, 6^, 8, and 9; 
5 more effective than 8 and 9; 7 more effective than 8 and 9. 

Conclusions: A*IN TERMS OF INSTRUCTIONAL EFFECTIVENESS, ECONOMY, AND SIMPLICITY 
OF PRODUCTION, THE RESULTS OF THIS STUDY INDICATED THAT? (1) THE SIMPLE LINE 
PRESENTATION WAS MOST EFFECTIVE IN FACILITATING ACHIEVEMENT ON THE DRAWING 
TEST: (2) THE DETAILED, SHADED DRAWING PRESENTATION (COLOR) WAS MOST EFFECTIVE IN 
FACILITATING ACHIEVEMENT ON THE IDENTIFICATION TEST; (3) THE ORAL PRESENTATION 
WITHOUT VISUALS WAS AS EFFECTIVE AS THE VISUALIZED TREATMENTS IN PROMOTING 
STUDENT ACHIEVEMENT ON THE TERMINILOGY AND COMPREHENSION TESTS: AND (A) THE 
SIMPLE LINE PRESENTATION (COLOR) WAS MOST EFFECTIVE IN PROMOTING STUDENT 
ACHIEVEMENT ON THE TOTAL CRITERIAL TEST. IT WAS FOUND THAT COLOR TREATMENTS 
WERE MORE EFFECTIVE THAN BLACK AND WHITE TREATMENTS IN 11 INSTANCES, AND IN 10 
INSTANCES, CERTAIN COLOR TREATMENTS WERE FOUND TO BE MORE EFFECTIVE THAN OTHER 
COLOR TREATMENTS.** 

000715 800 EVALUATION~The authors failed to mention whether the Instructional 
materials were given to groups or individuals. In addition the title of the 
article did not indicate that this study was concerned with the effectiveness of 
lifferent types of illustrations, as well as the effectiveness of color. Useful 
explanations of the various findings were included in the author's discussion. 
Sxcept for the author's occasional acceptance of his null hypotheses, this 3tudy 
ijas basically sound and has important implications for ISD in that it directly 
related types^of instructional tDaterials to specific types of criterion tests, 
however, no suggestions for future research were given. (R»A2, P-18, EMW) 
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• tfocumw>i dMcriAM to'^ttoM 
Ifrw^iUjl eighty . 

f J 0«ium^t 0o«« hot r«AMh MKV i^M pr1rKlp44«, 
la O^u'^^^At »f«««At« eertc«oti Mitvmt 10 
J J ttYlt »* ^itlPf l»«»|>roo,<«t, ,YPkH (SO i*mt. 

i% ot .h* doc^rn^, ,^ ^.^t,,^ linp«rH.i «cr«nmk ittNwdc 

JC^ALUATION CHECKUST FOR TVPE II ABSTftACTS 

». ^rcw«m i| itan^Kj iPtirty. 

». L'm rttic»«« o* ttudv tr* nitcd. 
*, Jm^iirtt t«rm( art d««n«d. 

>. Jf^•1^r^^^0 Ot iHt WObf#m |0 IX«vlOui rm^H U«Im». 

_ 11, M^od of lor^prlni !• •doqgw.ly dl«cr1fe0< ifMl N iM>^fl4Htnm 
' ' »r*'« ^ ido^ulioiv dowribod M to f loei»(oft. %M kM othor 

3* rr^> * ^►«ik>n »f ihi p<obiofn, 

^ *^ ^ -^ 12* *'v*oo*«d*.^t vfr.tiM irt hoi co^roo«^M n^ttf^ th« 

I ' j t m , — , I' *H»d, «>tp §n»ou IV w'^bMc 

I ■ - ■ ■ *^ mottvitloc it* fcrl»tol»i. 

I ' t I* Coftiro? I't iOOfOpriita coiT^lMn ^Ith Mp4rlnw»t>l 

- ' fOtp*. 

ai. Var'rt^^ tr. ot.*.,iohoJlt d«*tn#<J. IQ«o«|H«lM^ •«Mr*l1y» 
«. JOproorlitt FMgrta | ,.1^,^ fo< d,c*>d««1 vlf t«6lM, 

JJD 34. Co«mtttpnt iindof wM«h OKporlmof^i eo«(k«tid or* ctMrly BOcrfM 
!■ ■ , a¥U>4tt4<S in lu^kto^i tfofoit. wrrwd 

"fl kJ ^••^^ <«Hf«« F| of oth«r ipotirk --T^-mri. 

Li , I W ett|.B»if>or1iv# n%tth<HH 0^ p<o««<Kjr«c oro dMCrlbod Tn cufftelof^i ««t«ll 
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INTER-OPPICE MEMO 



Prom: M. W. Nesbltt. S. D. Colwell 

To: MT/ST Typists 

Subject: MT/ST Typing Requirements for Abstracts 

Date: 29 August 1972 



Conversion of the MT/ST tapes to computer-readable magnetic tapes 
iflposes certain requirements on the MT/ST typing. The rules which must 
be followed are outlined here. 



Material To Be Typed 

Only the second page of the abstract will be put on tape. The four 
data elements ares 

1. The bibliographic reference 

2. The abstract 

3. Tha evaluation 

4. The number of references (R« ) and pages (P« ), and the 
abstractor's initials 



Tape Preparation 



Before placing a tape cartridge on the tape station hub» the tape 

must be taut to preclude winkling or improper loading. This is 

accomplished by placing the fingers inside the inner ring and turning 
the cartridge as far as possible until the tape tightens. 

Every fifth tape should be checked for misalignment, using the 
following procedures; 

1* I>raw out 2 feet of tape. 

2. Apply chemical fluid supplied by IBM to tape surface. 
3* Using a magnifying glass » check that characters are 
centered within the spaces of the tape. 



special Codes Required 
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Each tape will begin with the same codes. These codes are necessary 
to allow the computer program to retrieve entire abstracts and/or data 
elemental where desirable « As shown in Figure 1» the codes consist ott 

Area A: A Peed Code and a Reference Codei then the S-digit 
Identifier (A1234) , another Reference Codei and two 
Carriage Returns. 

Area Bs A sequential tape number > beginning with a 0001, a 
space* a Stop Transfer Code, a Reference Code> an 
underscore, the digits '*01" and a space. 

Area C: The abstract sequence number, beginning with OOOOOi, 
and a r^pace* 

Area Ds An access number (600) and then a space before the 

bibliographic reference. This same number, 600, will 
be inserted before every bibliographic reference* 
The reference will then be typed, using quotes 
instead of an underscore for the document titles A 
Stop Transfer Code, a Reference Code, and two Carriage 
Returns will be typed. 

Area E; The abstract sequence number is repeated, a space, 

and then the access ntimber (700) is typed indicating 
the body of the abstract. The same number, 
will be inserted before every abstract texti (If 
the abstract contains experimental results, refer to 
the section on "Special Symbols.'') After the last 
line of the abstract, a Stop Transfer Code, a Refer- 
ence Code, and two Carriage Returns are inserted ■ 

Area F: The abstract sequence number is repeated, then a 
space > and then the access number (800) is typed 
indicating that the evaluation follows. This same 
number, 800, will be inserted before every evalu- 
ation. After typing the final entries (R-?, ?■»?, 
XXX), a Stop Transfer Code, a Reference Code, and 
two Carriage Returns are inserted. 

The typing of this abstract Is now completed. The next sequence 
number will be entered, and the procedure is repeated again as shown in 
Figure 1, where sequence number 000009 is the last abstract. When the 
end of the tape is reached, the procedure for ending the file Is as^ 
follows: ^-"^ 

' / 
1. When error light comes on, signaling the end of tape, the 
typist must line return twice for each abstract on the 
tape (e.g., 3 abstracts, 6 lines; 4 abstracts, 8 lines). ^ 
A maximum of 10 Line Returns should be made for tapes on 
which 5 or more abstracts are typed. The typist should 
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Beglnntngof Tap« 


Tape Numb«f j 






A 


1 


i 


9 


4 


RC 


CR 


CR 


0 


0 


0 


1 


SP 


STC 


RC 



'-®- 



Sequence Number 



SP 



Aeceit Cod« 



SP 



Bibtioaraphic Reference 
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RC 
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SMiuenM Numb* 
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Abitract Text 


TAPE 
1 


0 


0 


0 


0 


0 


1 


SP 


7 


0 


0 


SP 




STC 


RC 



CR CR 





Sequence Number 
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Evaluation 


0 


0 


0 


0 


0 


1 


SP 


8 


0 


0 


SP 




SP 


s^ 


# 


STC 



TAPE 
2 
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RC CR CR 



Sequence Number 


Accets Code 


Bibliographic Reference / 


0 


0 


0 


0 


0 


2 


SP 


6 


0 


9 


SP 


M ■ ^ 


STC 





/ Sequence Number 


Accets Code 


A'ystract Text / 




0 


0 


0 


0 


9 

._ 


SP 


7 


0 


'o 


SP 


"* " ' \ 



i End of File Code 


Unused Abstract Text 


# 


RC 


SC 


SC 


SC 








Beginning of Tape 


Tape Number 


FC 


RC 


A 


1 


2 


3 
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RC 


CR 


CR 


0 
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SP 


STC 


RC 



Sequence Number 


Access Code 


Abstract Text* 
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0 


0 


0 


0 


9 


SP 


7 
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0 


SP 
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STC 


RC 



CR CR 



Sequence Number 


Acceu Code 


Evaluatlort / 


0 
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SC 
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SP 


M ^ 


STC 





*6egin wtth unused text from Tape 1. 



Figure 1. Sample MT/ST Tape 
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line return to a period, so that the body of the abstract 
on that tape will end with a complete sentence* Begin the 
End of File Codfe at that point. This extra space will 
allow the tape to be coded at a later date for access by 
the computer. 

2» Area G of Figure 1 Illustrates the proper End of File 

Code» First » a pound sign (//) is keyed, then a Reference 
Code and three Stop Codes, followed by a slash 

3. VJhen starting a new tape, the typist must be careful about 
the initial codings The first codes from Ai^a A are the 
same, but the tape number is Increased by 1 (eigi, to 0002). 
If a new abstract is being started, use the new seouence 
number and proceed as if it were the first tape. If 4 
portion of the abstract (reference, abstract, or evalu^ 
ation) is continued from the previous tape % the access 
code changes from 600 (bibliographic reference), 700 
(abstract text), or 800 (evaluation) to 650, 750, or 850, 
indicating to the computer that this data is to fol.low the 
earlier element* (See Area H in Figure 1.) Each of the 
basic codes will be Increased by SO if they are U3ed to 
designate an item which has been split in this way. It is 
unlikely, however, that the bibliographic reference code 
would e^'er ne^d f-o b^ repeated, since references are 
typically quite short. 



Special Typing Requirements 

Unused Buttons 



The two MT/ST buttons which cannot be used on the abstracts are the 
Backspace Code (BC) button and the Automatic Search Code (ASC) button. 
Data recorded will not contain underscored words or centered headings, 
so the^BC will not be necessary. Also, it will not be acceptable to 
hyphenate words, so the BC will not be needed for this procedure. The 
ASC button cannot be used since the computer tape cannot be searched by 
this means. 



Typing at End of Line 

The Carrier Return (CR) Key records a Peed Code and a space* If a 
CR immediately follov;s a space, the output will be two spaces. For this 
reason, when a space would normally be recorded at the end of the typing 
line (followed by the CR) , record instead only the CR» This rule will 
not hold if a colon or period falls at the end of the line. 
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No indentions will be used, even for numbered Items. All lines will 
be left-hand justified. 

Reference Codes will not be used for each paragraph in the Type I 
abstracts* Howjver, Type II abstracts will receive R^iference Codes for 
each paragraph with a major heading (e.g.> Method, Results) to facilitate 
computer retrieval of data* 

To facilitate the computer's spacing, an extra space must be entered 
before the last Carrier Return in each paragraph. 



Correctinf^ Errors 

The Feed Code (FC) which advances the tape to block out unwanted 

characters must be used with discretion on computer input data. If more 

than five or six characters are Incorrect, use the Line Return procedure 

for corrections instead of Feed Ceding the characters out. 



Conclusions (Type II Abstracts) 

Where experimental conclusions appear in a bracketed sentence on the 
handwritten copy; thQ apjat'i'nce should be preceded by two asterisks (**) i 
typed in all caps, and followed by two asterisks (*f*) » For example: 

The authors concluded that **METHOD OF PRESENTATION DOES 
AFFECT THE READABILITY OF THE MATERIAL ON THE GRAPHS, FIGURE 
IDENTIFICATION, AND SYMBOL TRANSLATION TESTS AND THAT OFFSET 
COPY GAVE BETTER READABILITY THAN EITHER NEGATIVE OR POSITIVE 
MICROFICHES ON THESE THREE TESTS*** 



Special Symbols 

Symbols which must be avoided in typing MT/ST tapes of the abstracts 

ares 

1. The pound sign (//) 

2. The cent sign (c) 

3. The exclamation point (I) 

Experimental results will often appear in the Type II abstracts, and 
may contain numerical symbols or symbols which require translation by the 
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typiat» to assure that they are acceptable by the computer. The follow- 
' lag list Illustrates the necessary changes* 



This 


Not This 






(IT 


> 


*GE» 




• LE« 




^ sum 




frequency 


f 


mean 


X 


Alpha 





General Typlug Instructions 



Margins 

The standard margins for the recording of abstracts are 15-90 • . 



Paragraphing 

No paragraph indentions will be used} all material will be left-hand 
justified. Typing will be single-spaced, with a double-space between 
paragraphs. Also, "EVALUATION—" will be. typed in ell caps and will 
precede the text of the evaluation. 



Listings 

All listings of two (2) lines or less will be strung out In the 
sentence with parentheses around the identifying number or letter* All 
first divisions will be numbered with Arabic numbers (1|;2 > 3/ etc*) and 
the next division, with lower case letters (a, bid etc.)* 



Sentence StiTucture 

When possible, use complete sentences to express information^ 



Paper 



Draft paper will be used for recording of abstracts. Special 601!^ 
Marathon paper will be supplied for final copy of the abstracts* 



Proofing 



At the end of a typing period i the typist should take all completed 
tapes^ log sheets, and abstracts from the station proofed. Labels 
should be prepared for each tape, a typed version Wthe log sheet completed, 
and the abstracts carefully proofed for any errors. It will then be that 
typist's duty to make all corrections to those tapes at the beginning of • 
her next^ typing period. This will prevent #n enormous proof ing-correcting 
process at the end of the contract. 



Administrative 



Log Sheets 

A log sheet will be prepared for each MT/ST tape and will contain the 
following information! 

1. Tape number* 

2 1 Sequence numbers of abstracts which appear on that tape. 
3. A listing of the reference codes and a description of what 
is found at each code* 

For all tapes, the description for reference codes 01 and 02 is 
standard and an "S'N.As^^renrlntBd on the «heet. For Type I abstracts the 
appropriate codes are! B (Sequence No.) for bibliographic reference, A 
fcj body of abstract, E for evaluation, and S for the standard End of File 
code* The Type II abstracts will be referenced in the same way, except 
that Al, A2, A3, etc., will be used for each paragraph of the abstract that 
contains a major heading. Figures 2 and 3 pr:*ont example log sheets for 
Type I and II abstracts respectively. Fretvjtncly , however, Type I and II 
abstracts will be found on one tape* 

A solid line should be drawn across the De^scription column showing 
the end of each abstract. If an abstract is continued to the next tape, 
this would be indicated by a dotted line. (See Figures 2 and 3.) 

A handwritten log sheet should be prepared as the recording is being 
done and a final sheet should be typed after a day's activities. Log 
sheets will be kept in the MT/ST drawers. 



MT/ST Tapes 

Memorex MT/ST tapes are numbered consecutively in boxes of five <5) 
and are kept in the MT/ST work area. As needed, each box (containing 5 
tapes) should be taken to the MT/ST for recording and returned to the 
original carton when completed. 
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Ad each tape Is completed i a label (blue) should be prepared which 
contains the following Infortnatlon: 

!• Contract No. (1884) 
2« Tape No* 

3« Abstract Sequence Nos. which appear on the tape 

The following Is an example of such a labels It should be placed 
on the face of the tape, just below the tape number. 



Abstracts No. 1 through 6 




TAPE NO. -1 
ABSTRACT SEQUENCE NO. 24 



THROUGH«30 



Code 



01 
02 
03 
04 
05 
06 
07 
08 
09 
10 
11 
12 
13 
14 
16 
16 
17 
18 
19 
20 
21 
22 
23 
24 
26 
'26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 

57 
38 
39 
40 



Oescrtption 



S 

s 



B (24) 

A 

E 



A 
E 



B (26) 
A 

E 



B (27) 

A 

E 



B (28) 

A 

E 



B (29) 
A 

E 



B (30) 
A 



fl»ferene« 
Code 



1 



41 
42 
43 
44 
46 
46 
47 
46 

49 

60 
61 
62 

63 
54 
56 
56 
57 
58 
69 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 
74 
76 
76 
77 
78 
79 



D««Cflption 



I Document (with Idst abstract continued on next tape) 

A/. - ■ 



Reforenc« 
Cod« 



TAPE NO.. 

ABSTRACT SEQUENCE NO._JlIL THROUGH. 



D«ioript1on 



Rif«rerK« 
Cod« 



DMcrlptlon 



01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14 

16 

16 

17 

18 

18 

20 

21 

22 

23 

24 

26 

26 

27 

28 

2d 

30 

31 

3? 
33 

34 

36 
37 
38 
39 
40 



/ 



/ 



A 

E 



.A 
K 



T732T 

A 

S 



B (33) 

A 

E 



B (34) 
A 

E 



B (35y 

A 

B 



B (36) 
A 

E 



41 

42 

43 

44 

46 

46 

47 

48 

49 

60 

61 

62 

63 

64 

66 

66 

57 

68 

69 

60 

61 

62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

76 

76 

77 

78 

79 



TAPgMO 5 
ABSTRACT SEQUENCE N0...,2i- THROUQH^Ja 



R«fer«nc« 
Coda 



Detcrlptlon 



R«ferenc< 
Code 



Omlptlon 



01 
02 
03 
04 
06 
06 
07 
08 
09 
10 
11 
12 
13 
14 
16 
16 
17 
18 
19 
20 
21 

23 
24 
26 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 



S 
S 



B (2A) 

Al 

A2 

A3 

A4 

E 



B (25) 

Al 

A2 

A3 
E 



B (26) 
Al 

A2 
A3 

E 



B (27) 

Al 

A2 

A3 

A4 

E 



B (28) 

Al 

A2 
E 



B (29) 

Al 

A2 

A3 

AA 

E 



B (30) 
Al 
A2 
A3 

t 



41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
61 
52 
63 
64 
65 
66 
67 
58 
69 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 
74 
75 
76 
77 
78 
79 



Pigute 3 ; Type It Document 



000700 600 Dwyer, F, M», Jr, Exploratory studies in the effectiveness of 
visual illustrations. ''AV Comiaunication Review^" 1970, '18," 235-249. 

000700 700 The studies reviewed in this article investigated the problem o£ 
which typss of visual instruction are most efficient in facilitating student 
achievement of different types of educational objectivels* Conflicting 
theories about the effectiveness of realistic visual aids were presented* 
The author described his own study> which tested the effectiveness of simple 
line drawings, detailed drawings, and realistic photographs of a heart when 
these illustrations accompanied a taped lecture on the functioning of the 
heart* He found that the simple line drawings were significantly more 
effective in promoting student achievement on drawing, identification, and 
total criterial tests.. In addition, an oral presentation was as effective 
as presentations with visuals on terminology and comprehension tosts. In a 
similar study using college students, oral presentation alone was as 
effective as visually complemented treatments on four out of five criterial 
measures. Several explanations were advanced to explain these results. 

Another study used th. same text and illustrations in the form of a 
programmed text. The results of this study indicated that the use of visual 
Illustrations to complement programmed ins tr^fition is an effective way to 
improve achievement of objectives measured by drawing, identification, and 
total criterial tests. 

An extension of the seMe study added color as an instructional variable. 
It was found that for specific objectives and for students in certain grade 
levels, color appears to be an important variable for improving student 
achievement* 

Three similar studies were reported, each using a different medlumj (1) 
television, (2) telebe/ya projector producing a front projection Image, and 
<3) projector producing a rear-screen image. The same results occurred in 
each study. The oral presentation without visuals was found to be as 
effective as the visually complemented treatments on four of the five 
criterial tests. The exception was the drawing test, for which the simple 
line presentation was significantly more effective. 

The author concluded the article with a list of guidelines for the production 
and use of visual illustrations used for instructional purposes. 

000700 800 EVALUATION— This article is relevant to ISD as it contained 
suggestions and guidelines for the preparation and use of visual materials. 
The research designs were described adequately and studies were presented 
in logical sequence. However, in the studies discussed, the researcher has 
often accepted his implied null hypothesis, translating a ''not significantly 
different than" into an "as equally effective as." (R-30, P-15, Eifl) 

{ . - ../...:■■ 



\ figure 4. Sample Type I Abstract; 
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000704 600 Houser, R. L. , Houser, E. J. , & Van Modfrana, A. P. Learning 
a motion and a nonmotion concept by motion picture versus slide presentation. 
'AV Comnunicatlon Review," "18," 425-430. 

000704* 700 This study was related to the area of selection of media for 
most effective teaching. The effect of presenting a basic stimulus 
characteristic via two media with differing ability to present that 
characteristic was investigated. The authors hypothesized that: (1) the use 
of motion picture film would facilitate the learning of a concept involving 
motion as a defining attribute; (2) there would be no differences in learning' 
between a motion picture and a slide presentation in the learning of a 
nonmotion concept} and (3) repetition would facilitate learning a new concept. 

Method: The Ss were 41 volunteers from education classes at a large 
mldwestern university. They were randomly assigned to four groups. The 72 
stimulus patterns used in this study were irregular geometric shapes which 
differed in shape, angularity, size, and activity. Each was generated by 
choosing at random six pairs of numbers corresponding to points in a matrix. 
Lines wer-^ drawn between the points in a clockwise direction according to 
the order In which the pairs were drawn. For 3^ of the figures, the sharp 
angles were smoothed out using a French curve. Figures were selected at 
random for the presentation of both motion and nonmotion concepts. 

There were four experimental groups with the following presentations: (1) 
5 motion and 5 nonmotion exemplars by motion picture; (2) 10 motion and 10 
nonmotion exemplars by motion picture; (3) 5 motion and 5 nonmotion 
exemplars by slides; and (4) 10 motion and 10 nonmotion exemplars by slides. 

The presentations on motion pictures were filmed on black-and-white 8mm. 
film. Each sequence lasted 10 seconds with motion (90-degree rotation about 
the geometric center) occurring between seconds 5-8, or with no motion 
occurring. The Ss were asked to learn to label the various stimulus 
sequences with the nonsense syllables FIK or GEF for nonmotion and motion 
sequences, respectively. The slide presentations were identical except that 
the motion occurred in two discrete 45-degree rotations at the fifth and 
seventh seconds of the sequence. The Ss similarly labeled these with FIK 
or GEF. 

The dependent measure were the number of motion and nonmotion concepts 
correctly identified. The two independent variables were the number of 
presentations and the visual medium used. 

Results: An ANOVA was performed on the means of the number of motion 
concepts correctly identified. The Ss with the motion picture treatment 
correctly Identified significantly more motion concepts than the slide 
treatnient group (Fi»65. 87; df-l/40; p .LT. .001). Tile effect for number of 
presentations and the interaction of treatment and number of presentations 
were not significant. The number of nonmotion concepts correctly identified 
was significantly higher for the motion picture presentations than for slide 
presentations (F«37. 54; df»l/40; p .LT. .001). The effect for number of 
presentations was also significant (F=6.58; df-1/40; p .LT. .05). More 
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nonmotlon concepts were correctly Identified when subjects received 20 
presentations than when subjects received 10 presentations. 

Conclusions: ^*IN THE CASE WHERE MOTION IS A DEFINING ATTRIBUTE OF A 
CONCEPT, MORE EFFECTIVE LEARNING IS ACHIEVED WHEN THAT fiONCBPT IS PRESENTED 
USING MOTION PICTURES THAN WHEN IT IS PRESENTED USING SLIDES <** 

000704 800 EVALUATION*-- This research would best serve as groundwork for 
future research. Ko mention was made of previous related research* The 
title was sotnewaht deceptive > since ^^concept Identification'' oeems to be 
more descriptive of the task than Is '^concept learning*'' This study has 
little applicability to ISD because of the nature of the task Ss were asked 
to perform* However, the Idea of using motion pictures to present concepts 
Involving motion I although obvious, Is useful for ISD* (R*"3, P*6, EMW) 
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